In this work, by utilizing the monitoring data of a gravity retaining wall in the upperreservoir of Yixing PumpedStorage Hydropower Station, the variations of retaining wall displacement and their influence factors were studied. After that, a numerical simulation was conducted to investigate the displacements of rockfill dam and retaining wall with the consideration of the coupled effect of temperature. Here, the rheological model and wetting model (CwDw) proposed by Shen were used to model the rheological deformation and wetting deformation of rockfill materials. The results show that the rheological deformation, wetting deformation, and environmental temperature are three main influence factors for the displacement of retaining wall in the operation period. In the early stage of dam operation, there was a displacement for the retaining wall occurred towards upstream, which was mainly attributed to the relatively larger rheological deviator strain for rockfill materials near this place. The wetting deformation of rockfill materials was caused by rainfall infiltration and led to the increase of displacement of retaining wall towards downstream. The cycling variations of displacement were related to the environmental temperature. In addition, the calculated displacement and earth pressure agreed well with measured values, which indicates that the numerical method used in this study was effective.