Aiming at the problem that the vibration signal of pump unit is susceptible to random error and noise interference caused by various factors, which affects the evaluation result of the health status of pump unit, the filtering methods of vibration signal of pump unit were analyzed and studied. Taking the vibration signal at the pump coupling of a pumping station from the East Route of the South-to-North Water Transfer Project as an example, filtering methods based on Batworth, Hanning window, and harmonic wavelet were used to filter the vibration signal of the pump unit. The comparative analysis results show that there are time delay and line of axis trajectory in the FIR results of filtering methods based on Batworth and Hanning window, indicating incomplete filtering, while the harmonic wavelet filtering can almost completely filter out the interference. The advantages of harmonic wavelet filtering are more obvious when the frequency range is smaller for a certain frequency band.