In the opinion of Alejandro Lopez-Lira, a finance professor at the University of Florida, huge language models could be effective for forecasting stock values.
He utilized ChatGPT to interpret news headlines to determine if they were positive or negative for a stock, and discovered that ChatGPT's ability to forecast the direction of the next day's returns was substantially better than random, he said in a recent unreviewed work.
The experiment gets to the heart of the promise of cutting-edge artificial intelligence: These AI models may exhibit "emergent abilities," or capabilities that were not originally envisaged when they were constructed, with larger computers and better datasets, such as those powering ChatGPT.
If ChatGPT demonstrates an emerging capacity to interpret headlines from financial news and how they may affect stock prices, it may jeopardize high-paying positions in the finance industry. Goldman Sachs forecast in a March 26 paper that AI could automate 35% of finance jobs.
“The fact that ChatGPT is understanding information meant for humans almost guarantees if the market doesn’t respond perfectly, that there will be return predictability,” said Lopez-Lira.
However, the experiment's specifics demonstrate how distant "large language models" are from being capable of doing many banking jobs. The experiment, for example, did not include target pricing or require the model to perform any math at all. Indeed, as Microsoft discovered during a public demo earlier this year, ChatGPT-style technology frequently invents numbers. Sentiment analysis of headlines is also widely used as a trading strategy, employing proprietary algorithms.
Lopez-Lira was shocked by the findings, which he believes indicate that professional investors aren't yet incorporating ChatGPT-style machine learning into their trading tactics.
“On the regulation side, if we have computers just reading the headlines, headlines will matter more, and we can see if everyone should have access to machines such as GPT,” said Lopez-Lira. “Second, it’s certainly going to have some implications on the employment of financial analyst landscape. The question is, do I want to pay analysts? Or can I just put textual information in a model?”
How did the experiment work?
Lopez-Lira and his colleague Yuehua Tang examined over 50,000 headlines from a data vendor on public equities on the New York Stock Exchange, Nasdaq, and a small-cap exchange in the experiment. They began in October 2022, after the ChatGPT data cutoff date, implying that the engine had not seen or used such headlines in training.
The headlines were then sent into ChatGPT 3.5, along with the following prompt: “Forget all your previous instructions. Pretend you are a financial expert. You are a financial expert with stock recommendation experience. Answer “YES” if good news, “NO” if bad news, or “UNKNOWN” if uncertain in the first line. Then elaborate with one short and concise sentence on the next line.”
They then examined the equities' performance on the following trading day. Finally, Lopez-Lira discovered that when informed by a news headline, the model performed better in almost all circumstances. He discovered a less than 1% chance that the model would do as well picking the next day's move at random as it did when influenced by a news article.
ChatGPT also outperformed commercial datasets with human sentiment scores. According to the researchers, one example in the paper displayed a headline about a corporation settling litigation and paying a fine, which had a bad attitude, but the ChatGPT reaction correctly reasoned it was actually positive news.
According to Lopez-Lira, hedge funds have approached him to learn more about his findings. He also stated that he would not be surprised if ChatGPT's capacity to anticipate stock movements declined in the future months if institutions began to integrate this technology.
This is because the experiment only looked at stock prices the next trading day, although most people would expect the market to have priced the news seconds after it became public.
“As more and more people use these type of tools, the markets are going to become more efficient, so you would expect return predictability to decline,” Lopez-Lira said. “So my guess is, if I run this exercise, in the next five years, by the year five, there will be zero return predictability.”