Aiming at the problem of poor pre-processing of monitoring data of dangerous rock mass in the past, a VMD-XGBoost-GRU combined model was proposed for predicting deformation of dangerous rock mass. The model firstly uses variational mode decomposition (VMD) and sample entropy theory to decompose the deformation data of dangerous rock mass into multiple sub-sequences, then uses extreme gradient lift (XGBoost) algorithm to extract important model factors to achieve feature dimensionality reduction, and finally predicts the deformation of dangerous rock mass through gated recurrent unit (GRU) neural network. Taking the dangerous rock mass on the steep wall of the right abutment of a hydropower station as an example, the VMD-XGBoost-GRU combined model was compared with three other models, namely BP, GRU and VMD-XGBoost-BP. The results show that the VMD-XGBoost-GRU combined model has high precision in predicting the deformation of dangerous rock mass, and can provide technical basis for the evaluation of the safe and stable state of dangerous rock mass.