We obtain a partial description of the totally geodesic submanifolds of a 2-step, simply connected nilpotent Lie group with a left invariant metric. We consider only the case that N is nonsingular; that is, ad xi: N --> J is surjective for all elements xi is-an-element-of N - J , where N denotes the Lie algebra of N and J denotes the center of N. Among other results we show that if H is a totally geodesic submanifold of N with dim H greater-than-or-equal-to 1 + dim J, then H is an open subset of gN* , where g is an element of H and N* is a totally geodesic subgroup of N. We find simple and useful criteria that are necessary and sufficient for a subalgebra N* of N to be the Lie algebra of a totally geodesic subgroup N* . We define and study the properties of a Gauss map of a totally geodesic submanifold H of N. We conclude with a characterization of 2-step nilpotent Lie groups N of Heisenberg type in terms of the abundance of totally geodesic submanifolds of N.