In this paper, we consider high order multi-domain penalty spectral Galerkin methods for the approximation of hyperbolic conservation laws. This formulation has a penalty parameter which can vary in space and time, allowing for flexibility in the penalty formulation. This flexibility is particularly advantageous for problems with an inhomogeneous mesh. We show that the discontinuous Galerkin method is equivalent to the multi-domain spectral penalty Galerkin method with a particular value of the penalty parameter. The penalty parameter has an effect on both the accuracy and stability of the method. We examine the numerical issues which arise in the implementation of high order multi-domain penalty spectral Galerkin methods. The coefficient truncation method is proposed to prevent the rapid error growth due to round-off errors when high order polynomials are used. Finally, we show that an inconsistent evaluation of the integrals in the penalty method may lead to growth of errors. Numerical examples for linear and nonlinear problems are presented.