In this paper we consider the scattering of a plane acoustic or electromagnetic
wave by a one-dimensional, periodic rough surface. We restrict the discussion to the case when
the boundary is sound soft in the acoustic case, perfectly reflecting with TE polarization in the
EM case, so that the total field vanishes on the boundary. We propose a uniquely solvable
first kind integral equation formulation of the problem, which amounts to a requirement
that the normal derivative of the Green's representation formula for the total field vanish
on a horizontal line below the scattering surface. We then discuss the numerical solution by
Galerkin's method of this (ill-posed) integral equation. We point out that, with two particular
choices of the trial and test spaces, we recover the so-called SC (spectral-coordinate) and SS
(spectral-spectral) numerical schemes of DeSanto et al., Waves Random Media, 8, 315-414,
1998. We next propose a new Galerkin scheme, a modification of the SS method that we term
the SS∗ method, which is an instance of the well-known dual least squares Galerkin method.
We show that the SS∗ method is always well-defined and is optimally convergent as the size
of the approximation space increases. Moreover, we make a connection with the classical
least squares method, in which the coefficients in the Rayleigh expansion of the solution are
determined by enforcing the boundary condition in a least squares sense, pointing out that
the linear system to be solved in the SS∗ method is identical to that in the least squares
method. Using this connection we show that (reflecting the ill-posed nature of the integral
equation solved) the condition number of the linear system in the SS∗ and least squares
methods approaches infinity as the approximation space increases in size. We also provide
theoretical error bounds on the condition number and on the errors induced in the numerical
solution computed as a result of ill-conditioning. Numerical results confirm the convergence
of the SS∗ method and illustrate the ill-conditioning that arises.