Twilight of the Stock Jockeys
Year after year, funds actively managed by professional analysts fail to outperform passive index funds. So how are professional stock pickers still a thing? And how long before their clock runs out?
A trader works on the floor of the New York Stock Exchange (NYSE) at the opening bell in New York City on April 2, 2025. (Charly Triballeau / AFP via Getty Images)
In the 2023 novel The Counting House by Gary Sernovitz, an unnamed protagonist decides to resign as the manager of a top university’s endowment. He’s fallen into a pit of despair and anxiety after his investments have failed to perform for a third straight year. While the massive fund he oversees had grown incrementally over his tenure, he now has to admit that the gains turned out to be “less than we would have generated through a sixty/forty stock-bond passive portfolio.”
In other words, despite his vast expertise in esoteric financial instruments and investment strategies and all the fraught sales calls and interviews with prospective money managers, he would have been better off putting it all in index funds. Still, even as he writes and deletes and rewrites numerous versions of his resignation letter, he remains fixated on the idea that he’s done some good for a venerable if flawed institution of higher education.
Stock picking is on the decline. “The inevitable has arrived,” the investment research company Morningstar announced early last year. That January, more assets were controlled by “passively managed” funds than traditional “actively managed” ones — a historic first that had been a long time coming. Passive funds, or index funds, are pools of money that are invested according to the rules of an established index. Historically, because the picks are made “automatically,” index funds were considered a pretty unsexy way to invest, and for decades after they were first introduced to the public in 1976 they comprised only a sliver of the market. But by the early 2000s, they had become a hit, taking off exponentially after the 2008 financial crisis.
You will not “beat the market” by investing in an index fund because you are essentially invested in the whole market, or some significant segment of it like “mid-cap US stocks” or “investment-grade fixed income.” Nevertheless, the return on a fund set up to automatically mirror the S&P 500 index would have seen a growth rate of about 546 percent over the past twenty years — a pretty healthy return on your investment, especially considering you wouldn’t have had to pay anyone to do research and select stocks based on a combination of the research and some intangible, magical “intuition.” Because they’re automated, the fees on index funds are negligible. Year after year, the active stock pickers on the whole fail to beat the passive funds.
Thus in this area of finance, a field famously obsessed with identifying and ruthlessly exploiting inefficiencies and whose self-conception centers around dynamism and innovation, a conspicuously suboptimal state of affairs exists: a class of professionals manages to capture incalculable wealth while consistently underperforming their cheaper competitors.
So then, how are they still operating? A book-length study titled Inertia: Purposeful Inefficiencies in Financial Markets by the researchers Yuval Millo, Crawford Spence, and James J. Valentine aims to find out.
Twilight of the Stock Jockeys
Inertia investigates the social dynamics undergirding the persistence of active funds, tragicomically itemizing the arsenal of defense mechanisms these players deploy as they watch their market share collapse against a tidal wave of passive investing. Long-form interviews with financial professionals across a range of roles and experience levels provide abundant material for their qualitative analysis, which centers around their notion of “purposeful inefficiencies” brought about by congealed social relations and pressures.
Using a framework established by the sociologist Pierre Bourdieu, the authors put forward three fundamental ways communities of practice respond under threat: “First, they reassert a strong identity; second, they disparage those who are threatening them; and, third, they frame certain practices as inaccessible to others, highlighting themselves as the only possible saviors.” Millo et al. find in the reaction of traditional fund managers against the rise of passive investing all three of these defenses, which they call “distinction work.”
“I think that a lot of the role of a sell-side analyst is almost like a public good,” one participant explains:
There is obviously the specific kind of functions you perform that can be priced and quantified and charged for. But you are kind of, who’s paying for streetlights? You need them, but if they’re all switched off, you suddenly find a lot more cost and accidents, and I think there’s an element to the sell-side role that is very much part of that public-sector, public good–type function.
From this perspective, traditional fund management serves an important role in price discovery, serving not just clients but the market, and by extension society as a whole. The endowment manager in The Counting House is a kind of buy-side analog for this common refrain across the industry.
Fund managers also reiterate to the researchers that not enough time has passed to know whether passive really outperforms active. Sure, passive looks good while the market is up, but when the other shoe drops and stocks slip, active strategies will win the day, they say: “Just wait and see what the bear market brings.” While on the surface this argument might sound convincing, since passive success does rely on overall market gains, this hypothetical situation has already been realized and studied: there have been actual downturns recently, and active funds have fared empirically worse in those too.
Disparagement of index fund investing goes back further than the period under consideration by Millo, Spence, and Valentine. In 2016, the research firm Sanford C. Bernstein published the melodramatically titled report “The Silent Road to Serfdom: Why Passive Investing is Worse Than Marxism.”
The authors of this report, whose livelihoods are not coincidentally reliant on continued demand for traditional fund management, claimed that since index funds don’t deliberately allocate capital to companies with the best prospects for profitability and growth the way active managers do, they essentially undermine long-term economic growth in what amounts to “Marxist” centralism. But as Philip Coggan points out in the Financial Times, the companies tracked by indexes are already established, publicly listed firms; they’re mostly not looking around for new investment the way younger companies are. And when that investment arrives, it tends to come from banks, not active fund managers.
Participants in Millo et al’s study insisted that “if investors wanted to be ethical, they would have to enlist the help of the active community.” For the purposes of the study, the authors acknowledge but set aside the fact that so-called environmental, social, and governance investing (ESG) as a beneficent agent of social progress has been thoroughly debunked. The participant comments are an attempt to frame a rival product as incapable of enforcing ethical business practices within its funds.
But passive fund providers such as BlackRock have famously and to the chagrin of many Republicans enacted ESG policies, and as long-term investors forced by the rules of an index to hold a particular stock, they are considerably more incentivized to intervene in the management of their portfolio companies. Meanwhile, the argument that fund managers should be doing more to influence the direction of individual companies seems to come into conflict with the complaint that passive investment amounts to “Marxist” interference in market forces.
“In a theoretically efficient market, this regime [of active fund management] would have been swept away and replaced by ideas that encourage better resource allocation,” note the authors. “That these ideas and those who embody them have managed to retain such prominent, well-remunerated positions in financial markets is indicative of the extent to which social relations are congealed in economic fields.”
This state of congealed social relations isn’t surprising — it merely reveals finance to be susceptible to the same social dynamics as any other field. In fact, the authors acknowledge that active investing, despite its underperformance, probably isn’t going away and will carve out a space in the market that will continue to attract certain types of clients.
Purposeful Inefficiencies
In unpacking one of their primary concepts, the authors of Inertia show how purposeful inefficiency is exemplified in other areas of finance. Investment banks that broker deals between institutional investors and companies looking to raise capital also produce swaths of lengthy reports analyzing particular companies, which they provide to clients as a separate service. The glaring conflict of interest is that the companies that the banks match with investors are the same companies on which they issue analysis and buy, hold, or sell recommendations. If an analyst recommends selling a stock, they will surely lose access to that company as a party to new deals, which is how the firm makes money.
The analysts interviewed in the study are frank that they would almost never issue a Sell recommendation even if their research showed the company tanking. One explained that he would simply settle on Hold, no matter how bad things looked. As a result, the reports are lethally biased and hardly get read, which the analysts know. “Everyone recognizes that the majority of the things that you publish are just totally worthless. Earnings notes? Forget about it. Nobody reads that. . . . I don’t know. So why is there so much? I don’t know. Just legacy patterns that we’ve adopted over the years.”
Meanwhile consensus numbers, widely circulated and studied as a crucial input to trading decisions, represent the average earnings forecast of analysts covering a particular stock. Viewed through the screen of a Bloomberg Terminal or other disinterested financial data platform, they appear to many players as objective indicators of “the market’s” opinion of a stock, when in fact they are biased by herding: participants explained how they would reconsider their predictions if they deviated from the existing consensus.
Institutional investors who rely on these figures to decide how to allocate money then amplify the average prediction even further by providing their own “clean-up,” removing perceived outliers among a set of predictions on the assumption that “there’s something wrong” for one reason or another. Like the case of the useless glut of biased company reports, this situation is widely recognized by the players involved — and yet the consensus numbers maintain their power. A lot of hours are put into bringing forecast figures in line or writing analyst reports, but little of value gets published.
There will always be clients attracted to the active fund’s potential for big gains, the same way people like to gamble on sports or speculative digital assets like Bitcoin. But we can drop the pretense that fund managers are performing some societal benefit or warding off worse outcomes through natural genius, gut instincts, and rigorous research, or that the financial industry is characterized by hard facts as opposed to the same ossification of ideas and relationships as any other professional field. Inertia is a fascinating and often bitingly funny showcase of the cognitive dissonance constantly grappled with by a diminishing class of financial professionals and a study of the odd, social side of finance.