This paper is intended to be a first step towards the continuous dependence of dynamical contact problems on the initial data as well as the uniqueness of a solution. Moreover, it provides the basis for a proof of the convergence of popular time integration schemes as the Newmark method. We study a frictionless dynamical contact problem between both linearly elastic and viscoelastic bodies which is formulated via the Signorini contact conditions. For viscoelastic materials fulfilling the Kelvin-Voigt constitutive law, we find a characterization of the class of problems which satisfy a perturbation result in a non-trivial mix of norms in function space. This characterization is given in the form of a stability condition on the contact stresses at the contact boundaries. Furthermore, we present perturbation results for two well-established approximations of the classical Signorini condition: The Signorini condition formulated in velocities and the model of normal compliance, both satisfying even a sharper version of our stability condition.