We establish a framework to study the factor structure in stock variance under a high-frequency and high-dimensional setup. We prove the consistency of conducting principal component analysis on realized variances in estimating the factor structure. Moreover, based on strong empirical evidence, we propose a multiplicative volatility factor (MVF) model, where stock variance is represented by a common variance factor and a multiplicative lognormal idiosyncratic component. We further show that our MVF model leads to significantly improved volatility prediction. The favorable performance of the proposed MVF model is seen in both US stocks and global equity indices.