In order to better solve the drop problem of cleaning robot caused by the lack of effective boundary detection and satisfy the requirement of full coverage cleaning, a navigation method based on the hierarchical cost map is proposed for the cleaning robot of photovoltaic(PV)systems. On the layer of the traditional hierarchical cost map, the boundary layer of the PV array is added to establish the cleaning area, according to the analysis of the working environment of the robot, so that the working space of the robot can be constrained in the PV array. Then, the navigation strategy combining the bow shape cleaning path and multi-point navigation is adopted to realize the full traversal of cleaning area. Moreover, the second round of cleaning judgment is introduced to remove the residual contaminants on the surface of PV array, which improves the cleaning efficiency of robot. The experimental results show that the proposed navigation method can solve the drop problem of robot due to the failure of boundary detection, and the robot can effectively complete the full coverage cleaning work.