We consider cosmological perturbations around FLRW spacetimes minimally coupled to a scalar field and present a formulation designed to preserve covariance. The perturbations are described in terms of gauge invariants, linear perturbative constraints, and variables canonically conjugate to them. This set is completed into a canonical one for the entire system, including the homogeneous degrees of freedom. We then explain how to quantize this system, respecting covariance and allowing for backreaction. Finally, by adopting a type of Born-Oppenheimer ansatz for the physical quantum state, we show how to extract equations of motion for the primordial cosmological perturbations that retain quantum geometry effects. The application to Loop Quantum Cosmology is almost direct.