We study the model consisting of scalar perturbations around a homogeneous and isotropic universe with compact spatial sections, filled by a massive scalar field. The local gauge freedom is fixed in two alternative ways, with consistent results. We construct a complete quantization of this model by combining a polymer representation for the homogeneous degrees of freedom and a Fock representation for the inhomogeneities. The latter is uniquey selected by the criteria of symmetry and unitarity of the dynamics in the limit of quantum field theory on a curved spacetime, and can be alternatively formulated in terms of gauge invariants. We also propose a quantum implementation of the Hamiltonian constraint and characterise their solutions.