To improve the accuracy of real-time flood forecasting in small and medium-sized basin in mountainous areas, a KNN real-time correction method based on the historical flood learning (KNN-H) was proposed and applied to two small basins in mountainous area in the Loess Plateau of Northern Shaanxi Province to test its performance. The proposed method was compared with the traditional KNN and AR methods. The results show that the correction accuracy of the KNN and KNN-H method is higher than that of the AR method and the traditional KNN and AR method cannot effectively reduce the peak time error of the forecasts, while the KNN-H method can reduce the peak time error. The correction accuracy of KNN is not high due to insufficient data in preheating period, while the KNN-H method effectively solves this problem by learning the historical flood forecast error. When the forecasted flood process is in the flood rising or falling stage, the KNN-H can locate the same stage of historical flood quickly and correct the current forecast value after analyzing the historical forecast error. In general, the correction accuracy of the KNN-H method is higher than that of the traditional KNN method.