We propose a polygonal finite volume element method based on the mean value coordinates for anisotropic diffusion problems on star-shaped polygonal meshes. Because the convex cells with hanging nodes are always star-shaped, the computation on them is no longer a problem. Naturally, we apply this advantage of the new polygonal finite volume element method to construct an adaptive polygonal finite volume element algorithm. Moreover, we introduce two refinement strategies, called quadtree-based refinement strategy and polytree-based refinement strategy respectively, and they all have great performance in our numerical tests. The new adaptive algorithm allows the use of hanging nodes, and the number of hanging nodes on each edge is unrestricted in general. Finally, several numerical examples are provided to show the convergence and efficiency of the proposed method on various polygonal meshes. The numerical results also show that the new adaptive algorithm not only reduces the computational cost and the implementation complexity in mesh refinement, but also ensures the accuracy and convergence.