To reflect the dilatancy behavior of the coarse-grained materials, the dilatancy equation in the form of the fourth power function was introduced to modify the basic hhu-SH model that is an elastoplastic constitutive model of coarse grains based on the mesoscopic structure. In addition, the yield function, hardening parameter, derivation process of initial elastic modulus, and parameter determination method in the modified hhu-SH model were given. Relying on the concrete-faced rockfill dam (CRFD) in a pumped storage power station, the parameters of the modified hhu-SH model for different coarse-grained materials were determined by using the results of CD triaxial tests and isotropic compression tests. Then, the numerical simulation was conducted to study the stress and deformation of CRFD under the action of filling and impounding. Meanwhile, the calculated results were compared with those by using Duncan E-B model and Shen’s elastoplastic model. It shows that the results obtained from modified hhu-SH model are similar to those of Duncan’s E-B model and Shen’s elastoplastic model in the respect of the distribution of dam deformation, which validates the applicability of the modified hhu-SH model. However, from a quantitative perspective, the deformation of dam, deformation and stress of the concrete-slab, and displacement of the joints calculated by the modified hhu-SH model were smaller than that by Duncan’s E-B model generally, while closer to that of Shen’s elastoplastic model. Furthermore, the stress of the concrete-slab along the slope is the compression stress generally, which is consistent with the monitoring.