In order to address the low statistical efficiency and accuracy of the number of armour unit blocks of slope breakwater, a method for the recognition and segmentation of the slope breakwater accropodes based the Mask R-CNN deep learning network was proposed. Firstly, this method used the Mask R-CNN network learning laboratory to collect the feature information of the image. Secondly, the model with the best performance evaluation index was obtained by adjusting the IOU threshold. Finally, the trained Mask R-CNN network was applied in the recognition and segmentation of the armour blocks of the on-site breakwater image. The test results show that when the IOU is 0.5, the average accuracy rate of target segmentation is 91.83% and the average recall rate is 92.94%. Using the trained model to detect the breakwater images taken by drones in actual projects, the identification rate of accropodes is 90.7%, and the shooting angle and height have little effect on the identification accuracy. Therefore, the Mask R-CNN deep learning network can realize the accurate recognition of dense and complex armour layer blocks with good portability and versatility.