Given a vector bundle F on a variety X and W subset of H-0(F) such that the evaluation map W circle times O-X -> F is surjective, its kernel S-F,S-W is called generalized syzygy bundle. Under mild assumptions, we construct a moduli space G(U)(0) of simple generalized syzygy bundles, and show that the natural morphism alpha to the moduli of simple sheaves is a locally closed embedding. If moreover H-1(X, O-X) = 0, we find an explicit open subspace G(v)(0) of G(U)(0) where the restriction of alpha is an open embedding. In particular, if dim X >= 3 and H-1(OX) = 0, starting from an ample line bundle (or a simple rigid vector bundle) on X we construct recursively open subspaces of moduli spaces of simple sheaves on X that are smooth, rational, quasiprojective varieties.