We present the orthogonality-constrained Hartree-Fock (HF) method for excited states in a combination with the Moller-Plesset-like perturbation theory for the correlation energy. This developed "HF + MP2" formalism for excited states allows for the treatment of both ground and excited states in a balanced manner. Unlike a previous work (Glushkov in Chem Phys Lett 287:189, 1998), our interest has shifted toward highly doubly excited states of atoms and doubly ionized core hole molecular states which are attractive from the experimental point of view. The accuracy of the method is demonstrated by calculations of more than 30 highly excited states of the He and Li atoms and about 10 doubly excited core hole states of some diatomic molecules (CO, NO and LiF).