In order to realize the rapid and accurate inspection of aggregate particle size in hydraulic engineering construction, an automatic aggregate image segmentation algorithm based on DeepLabV3+ was proposed. 150 aggregate images under different conditions were collected, and the network was optimized based on the original DeepLabV3+ network through contrast experiment. Then the optimized network was used to train the automatic aggregate image segmentation model. The MobileNetV2 is the backbone network of the improved DeepLabV3+ network, and the Swish+BN function is the activation function.After weight optimization, the aggregate’s intersection over union (IoU) is 0.861 5, which is 0.011 8 higher than the original network training model, and 0.064 6 and 0.088 6 higher than that of U-Net and FCN training models. The automatic segmentation accuracy of aggregate image based on the improved DeepLabV3+ can basically meet the accuracy requirements.