This study introduces an innovative approach for constructing multimodal investor sentiment indices and explores their varying impacts on stock market returns. We employ the RoBERTa model to quantify text-based sentiment, the Google Inception(v3) model for image-based sentiment measurement, and a multimodal semantic correlation fusion model to comprehensively consider the interplay between textual and visual sentiment features. These sentiment indices are further categorised into industry-specific investor sentiment and market-wide investor sentiment, enabling separate analyses of their effects on stock markets. Furthermore, we leverage these indices to build a multifactor stock selection model and timing strategies. Our research findings demonstrate that multimodal sentiment analysis yields superior predictive accuracy. Industry-specific investor sentiment exerts bidirectional positive influences on stock market returns, whereas market-wide investor sentiment indices exhibit unidirectional impacts. Integrating industry-specific investor sentiment into our multifactor stock selection model effectively enhances portfolio returns. Furthermore, combining market-wide investor sentiment with timing strategy optimisation further augments this advantage.

Behavioral theories suggest that investor misperceptions and market mispricing will be correlated across firms. We use equity and debt financing to identify common misval- uation across firms. A zero-investment portfolio (UMO, undervalued minus overvalued) built from repurchase and issue firms captures comovement in returns beyond that in some standard multifactor models, and substantially improves the Sharpe ratio of the tangency portfolio. Loadings on UMO incrementally predict the cross-section of returns on both portfolios and individual stocks, even among firms not recently involved in external fi- nancing activities. Further evidence suggests that UMO loadings proxy for the common component of a stock’s misvaluation.

There is mixed evidence of a positive relationship between the stock market risk and return. We reexamine this critical implication of asset pricing theory using fresh data from China's stock market, which is largely segmented from the rest of the global financial market. Using formal variable selection methods and a comprehensive set of predictor variables, we identify conditional market variance, scaled market prices, and inflation as crucial determinants of equity premiums. The estimated simple risk-return relationship exhibits downward omitted variable bias, which underlines the importance of considering multiple factors to explain the variation in equity premiums. We cannot wholly attribute the three-factor conditional equity premium model to data mining, as Guo, Sanni, and Yu (2022) select the same model for the U.S. stock market. These findings challenge existing asset pricing models and provide valuable guidance for future theoretical research.