In this paper we consider the computation of some eigenpairs with smallest eigenvalues in modulus of large-scale polynomial eigenvalue problem. Recently, a partially orthogonal projection method and its refinement scheme were presented for solving the polynomial eigenvalue problem. The methods preserve the structures and properties of the original polynomial eigenvalue problem. Implicitly updating the starting vector and constructing better projection subspace, we develop an implicitly restarted version of the partially orthogonal projection method. Combining the implicit restarting strategy with the refinement scheme, we present an implicitly restarted refined partially orthogonal projection method. In order to avoid the situation that the converged eigenvalues converge repeatedly in the later iterations, we propose a novel explicit non-equivalence low-rank deflation technique. Finally some numerical experiments show that the implicitly restarted refined partially orthogonal projection method with the explicit non-equivalence low-rank deflation technique is efficient and robust.