This study was carried out in the basin located above the Xixian Hydrological Station in the upper reaches of the Huaihe River. A grid-based(1 km×1 km)Xin’anjiang model was coupled with the conceptual model of soil erosion to build the sediment load simulation model for the upper reaches of the Huaihe River. Using the daily measured data of rainfall, runoff, and sediment concentration at the Xixian Hydrological Station from 2000 to 2008, the sediment load in the basin above the Xixian Hydrological Station was calculated. The results show that the built model can be effectively used to simulate the sediment yielding process in the Huaihe Basin, and the model has high precision. This study provides a scientific basis for the control of water loss and soil erosion in the upper reaches of the Huaihe River and provides references for the simulation of water loss and soil erosion in other basins.