Finite volume method with first order and second order Godunov schemes is presented to simulate and analyze the water hammer problem. Firstly, the Riemann solver is used to solve the discrete flux and the convection term is considered. MUSCL-Hancock method is used to reconstruct the second order precision Godunov scheme, while the MINMOD slope limiter is introduced to avoid spurious oscillations. The virtual boundary method is proposed to deal with the boundary calculations, which can realize a unified calculation scheme for all the nodes inside the computational domain and at the boundaries. Analysis shows that the virtual boundary method can ensure the accuracy of the calculation results, and it can also avoid the complexity of the boundary computation. The results predicted by the first-order Godunov scheme are identical with those from the Method of Characteristics(MOC). When the Courant number is less than one, the transient pressure calculated from the first-order Godunov scheme and the MOC scheme may appear serious numerical dissipation. However, the second-order Godunov scheme can effectively suppress numerical attenuation and obtain more accurate and stable results. In the cases with a low Maher number, the convection term can be neglected in simulations.