This work presents a set of preconditioning strategies able to significantly accelerate the performance of fully implicit energy-conserving Particle-in-Cell methods to a level that becomes competitive with semi-implicit methods. We compare three different preconditioners. We consider three methods and compare them with a straight unpreconditioned Jacobian Free Newton Krylov (JFNK) implementation. The first two focus, respectively, on improving the handling of particles (particle hiding) or fields (field hiding) within the JFNK iteration. The third uses the field hiding preconditioner within a direct Newton iteration where a Schwarz-decomposed Jacobian is computed analytically. Clearly, field hiding used with JFNK or with the direct Newton-Schwarz (DNS) method outperforms all method. We compare these implementations with a recent semi-implicit energy conserving scheme. Fully implicit methods are still lag behind in cost per cycle but not by a large margin when proper preconditioning is used. However, for exact energy conservation, preconditioned fully implicit methods are significantly easier to implement compared with semi-implicit methods and can be extended to fully relativistic physics.