Exploring the Synergy of Quantitative Factors and Newsflow Representations from Large Language Models for Stock Return Prediction

In quantitative investing, return prediction supports various tasks, including stock selection, portfolio optimization, and risk management. Quantitative factors, such as valuation, quality, and growth, capture various characteristics of stocks. Textual data, like news and earnings transcripts, has attracted growing attention, driven by recent advances in large language models (LLMs). This paper examines effective and practical ways to leverage multimodal factors and newsflow for return prediction and stock selection. First, we introduce a fusion learning framework to learn a unified representation from factors and newsflow representations generated by an LLM. Within this framework, we compare three methods of different architectural complexities: representation combination, representation summation, and attentive representations. Next, building on the limitation of fusion learning observed in empirical comparison, we explore the mixture model that adaptively combines predictions made by single modalities and their fusion. To mitigate the training instability of the mixture model, we introduce a decoupled training approach with theoretical insights. Finally, our experiments on real investment universes reveal: (1) Within fusion learning, the representation combination method, despite its relatively low architectural complexity, generally outperforms other fusion methods. The relative performance of fusion methods varies across investment universes, suggesting variation in the predictive relevance of data, particularly news. (2) The mixture model appears relatively robust across universes and portfolios, delivering comparable or superior performance. Its enhanced adaptability can be beneficial in settings where the predictive relevance of factors and news is likely more variable. (3) Fine-tuning the LLM during the training of these multimodal models does not consistently benefit performance; its impact varies across universes, potentially reflecting differences in market efficiency and data characteristics.