We give a covariant characterization of the Penrose plane wave limit: the plane wave profile matrix A(u) is the restriction of the null geodesic deviation matrix (curvature tensor) of the original spacetime metric to the null geodesic, evaluated in a comoving frame. We also consider the Penrose limits of spacetime singularities and show that for a large class of black hole, cosmological and null singularities (of Szekeres-lyer 'power-law type'), including those of the FRW and Schwarzschild metrics, the result is a singular homogeneous plane wave with profile A(u) similar to u(-2), the scale invariance of the latter reflecting the power-law behaviour of the singularities.