Will artificial intelligence take over quantitative investing?

Machine learning based artificial intelligence models seem like natural partners for quantitative investing — which often uses complex mathematical and statistical models to identify opportunities to buy and sell securities.

By its very nature, quant investment strategies rely on the analysis of large amounts of data. So do AI/ML models. In fact, the lines are blurring between quantitative approaches and AI/ML approaches to investing as these techniques become more mainstream.

Both traditional quantitative and AI/ML models attempt to make predictions from patterns in market, economic and company-related data. Both seek to identify correlations. Both are most successful when they can identify causal relations. And both are becoming essential tools for investors.

At many brokerages and asset managers, quantitative strategies are used for all asset classes, often to complement more fundamental strategies. AI is growing in popularity, too. For example, a 2019 survey by the Bank of England and the UK’s Financial Conduct Authority found that two-thirds of all British financial firms were already using machine learning¹.

But there are limitations. While the synergies are fairly obvious, AI’s application for process automation and marketing has been more popular than for picking stock winners. A 2019 survey by the CFA Institute found that only 10% of portfolio manager respondents had used AI/ML techniques in the previous 12 months². There remain skeptics as to how to incorporate AI methods into an investment process when the output of many modeling approaches may not make intuitive sense from a human investor’s standpoint. How does one explain performance or positioning outcomes to clients when the decision was made by a machine?

In the short term, while AI is certainly an exciting area of discovery and is helping to push the boundaries on quantitative techniques, it is less likely to replace quant managers than to work alongside them, helping come up with better predictions based on increasingly large, non-linear, and new (alternative) categories of data sets.

Fig 1: Quant vs AI/ML strategies

Quantitative investment strategies are already heavily dependent on crunching numbers. Fundamental and macro strategies also may rely on large amounts of data to draw investment conclusions, whether in the form of economic, stock market or company-specific information. One way AI can help is simply by helping to filter out signals or ideas that are less efficacious, thereby allowing time to be spent on what matters most to the investment process.

AI also allows for the processing of more types of data (structured and unstructured) as well as much larger volumes. Machines can be taught to read profit statements and summarise executive sentiment, or scan audio from press conferences, or to look for particular items in images, videos, or weather reports. It can also use non-numerical data like satellite imagery, flight and shipping information, and social media content among other data sets – paving the way for the holy grail of quant investing – new, undiscovered and uncorrelated alpha factors.

Because AI/ML models excel at finding patterns in data, it can potentially enhance traditional data-driven investment strategies by improving the interpretation of that data and applying it to investment decisions. Furthermore, these models are not limited by the imagination of human researchers in identifying drivers of returns or of any relationship in the data. It allows humans to discover new patterns or unique insights that may have been overlooked as they are not obvious.

AI can also enhance risk management by identifying complex correlated exposures missed by typical risk models. When built the right way, rules-based AI systems are less prone to mistakes than humans, because algorithms operate without the biases and emotions that can unintentionally compromise portfolio managers.

And biases are extremely important. In fact, a recent study in the Harvard Business Review that tested an algorithm against angel investors found that the algorithm achieved an average internal rate of return (IRR) of 7.26%, while the 255 angel investors yielded average IRRs of 2.56%.³ The main reason was bias in the decision making process.

There were, however, a group of elite investors who achieved an average IRR of 22.75%, mostly because they were able to suppress their biases and rely on the data, but also had enough experience and intuition to spot opportunities that the algorithm missed.

One significant limit on AI’s capabilities is the willingness of organisations to adopt it. Large datasets are complicated to manage. They need to be combined, standardised, cleaned of outliers, and turned into useful insights. And while using AI in an investment strategy can help to eliminate biases, AI is only ever as good as the data it relies on. Flawed data will yield poor results.

For example, at Eastspring Investments, we have leveraged non-traditional data in the form of quarterly corporate earnings call transcripts using Natural Language Processing (NLP), a subset of machine learning. However, the raw data is far from perfect and requires significant effort. We are also able to fine-tune our existing, tried-and-tested factor strategies by using AI techniques to help weight factors in line with market trends.

AI usually requires specialised skillsets, which might be costly and difficult to recruit. AI could also throw up potential issues around data ownership and process transparency. So perhaps it is not surprising that many businesses feel they have good reason to proceed cautiously.

But there’s a more fundamental issue. While most people think of stock markets as a data rich environment, in truth there are data issues. Even in the US, where stock market statistics are most complete and date back to the 1920s, quality fundamental data has only become available since the 1960s. Elsewhere, the volume and quantity vary considerably. For AI to work effectively, the models need many observations; the more observations the better for large neural networks to be trained properly.

Worse still, the available data for some specific (but very important) types of events is particularly thin. Many AI-focused models fared particularly poorly in March last year, when the Covid-19 pandemic sent markets into a tailspin.⁴ The main reason was the heightened volatility that had only been seen a handful of times previously. Accordingly, algorithms couldn’t make sense of market conditions or offer useful insights in such a unique environment.

AI is likely to dramatically reshape the workforce and the wider economy over the coming years, as businesses make use of new technologies to massively increase their productivity. However, machines are still a long way from replicating some human thought processes in the sphere of investment management.

It is unlikely that investment managers will be replaced by machines in the foreseeable future. It is far more likely that investment professionals will work closely with AI tools and techniques. The best results may come from letting a machine learn from expert investors and using the algorithm to extend the breadth of coverage.

Asset managers will spend more time developing strategies, identifying new data sources, updating algorithms, and interpreting their results. Using AI/ML models to complement existing ones will ultimately help to better manage portfolio risk in client portfolios.