In this paper, we study an exponential time differencing method for solving the gauge system of incompressible viscous flows governed by Stokes or Navier-Stokes equations. The momentum equation is decoupled from the kinematic equation at a discrete level and is then solved by exponential time stepping multistep schemes in our approach. We analyze the stability of the proposed method and rigorously prove that the first order exponential time differencing scheme is unconditionally stable for the Stokes problem. We also present a compact representation of the algorithm for problems on rectangular domains, which makes FFT-based solvers available for the resulting fully discretized system. Various numerical experiments in two and three dimensional spaces are carried out to demonstrate the accuracy and stability of the proposed method.