Exotic quantum phase transitions in a Bose-Einstein condensate coupled to an optical cavity

An extended Dicke model, which includes atom-atom interactions and a driving classical laser field, is established for a Bose-Einstein condensate inside an ultrahigh-finesse optical cavity. A feasible experimental setup with a strong atom-field coupling is proposed, where most parameters are easily controllable and thus the predicted a second-order superradiant-normal phase transition may be detected by measuring the ground-state atomic population. More intriguingly, second-order phase transition from the superradiant phase to the "Mott" phase is also revealed. In addition, a rich and exotic phase diagram is presented.