We construct an efficient numerical scheme for the quantum Fokker-Planck-Landau (FPL) equation that works uniformly from kinetic to fluid regimes. Such a scheme inevitably needs an implicit discretization of the nonlinear collision operator, which is difficult to invert. Inspired by work [9] we seek a linear operator to penalize the quantum FPL collision term Q_qFPL in order to remove the stiffness induced by the small Knudsen number. However, there is no suitable simple quantum operator serving the purpose and for this kind of operators one has to solve the complicated quantum Maxwellians (Bose-Einstein or Fermi-Dirac distribution). In this paper, we propose to penalize Q_qFPL by the "classical" linear Fokker-Planck operator. It is based on the observation that the classical Maxwellian, with the temperature replaced by the internal energy, has the same first five moments as the quantum Maxwellian. Numerical results for Bose and Fermi gases are presented to illustrate the efficiency of the scheme in both fluid and kinetic regimes.