We aim at a fast finite element method for the Poisson equation in three-dimensional infinite domains. Both the exterior and strip-tail problems are considered. By introducing a suitable artificial boundary and imposing the exact boundary condition of Dirichlet-to-Neumann (DtN) type, we reduce the original infinite domain problem into a truncated finite domain problem. The point is how to efficiently implement this exact artificial boundary condition. The traditional modal expansion method is hard to apply for the strip-tail problem with a general cross section. We develop a fast algorithm based on the Padé approximation for the square root function involved in the exact artificial boundary condition. The most remarkable advantage of our method is that it is unnecessary to compute the full eigen system associated with the Laplace-Beltrami operator on the artificial boundary. Besides, compared with the modal expansion method, the computational cost of the DtN mapping is significantly reduced. We perform a complete numerical analysis on the fast algorithm. Some numerical examples are presented to demonstrate the effectiveness of the proposed method.