The p-rank strata of the moduli space of hyperelliptic curves

We prove results about the intersection of the p-rank strata and the boundary of the moduli space of hyperelliptic curves in characteristic p >= 3. This yields a strong technique that allows us to analyze the stratum H-g(f) of hyperelliptic curves of genus g and p-rank f. Using this, we prove that the endomorphism ring of the Jacobian of a generic hyperelliptic curve of genus g and p-rank f is isomorphic to Z if g >= 4. Furthermore, we prove that the Z/l-monodromy of every irreducible component of H-g(f) is the symplectic group Sp(2g) (Z/l) if g >= 3, and l not equal p is an odd prime (with mild hypotheses on f when f = 0). These results yield numerous applications about the generic behavior of hyperelliptic curves of given genus and p-rank over finite fields, including applications about Newton polygons, absolutely simple Jacobians, class groups and zeta functions. (C) 2011 Elsevier Inc. All rights reserved.