Machine-learning stock market volatility: Predictability, drivers, and economic value

We investigate whether machine learning (ML) techniques, using a large set of financial and macroeconomic variables, help to predict S&P 500 realized volatility and deliver economic value. We evaluate regularization methods (Ridge, Lasso, and Elastic Net), tree-based methods (Random Forest and Gradient boosting), and Neural Networks. We find that ML algorithms outperform the benchmark model (HAR) at a short horizon (1 month), but not over longer periods (6 and 12 months). Regularization methods and Neural Networks emerge as the most competitive ML methods. We find that the quality of predictors is crucial, with financial and macroeconomic uncertainty proxies playing the most significant role. From an economic perspective, however, predictive ML models do not yield substantial gains compared to the benchmark. •We fit machine learning methods to aggregate stock market volatility.•We use a large set of predictors (big data).•ML models beat the benchmark model in the short run but not the long run.•Regularization models and Neural networks perform the best.•ML models do not deliver economic gains compared to the benchmark.