We consider a diffuse interface model for the flow of two viscous incompressible Newtonian fluids with different densities in a bounded domain in two and three space dimensions and prove existence of weak solutions for it. In contrast to earlier contributions, we study a model with a singular nonlocal free energy, which controls the H-alpha/2-norm of the volume fraction. We show existence of weak solutions for large times with the aid of an implicit time discretization.