Correlations in the illumination field have a profound impact on the image contrast for features near the resolution limit. The pupil polarization affects these correlations. We show that a polarization vortex has a particularly dramatic effect. A theoretical model is given for the correlation matrix of a partially correlated source created by placing an azimuthal polarization vortex mode converter in the pupil plane of a critical illumination system. We then validate this model experimentally using a reversed-wavefront Young interferometer, directly show the impact that the phase of the correlation function has on image contrast. (C) 2008 Optical Society of America