Vortex formation in a fast rotating Bose-Einstein condensate

We study rotational motion of an interacting atomic Bose-Einstein condensate confined in a quadratic-plus-quartic potential. We calculate the lowest energy surface mode frequency and show that a symmetric trapped (harmonic and quartic) Bose-Einstein condensate breaks the rotational symmetry of the Hamiltonian when rotational frequency is greater than one-half of the lowest energy surface mode frequency. We argue that the formation of a vortex is not possible in a noninteracting as well as in an attractive Bose-Einstein condensate confined in a harmonic trap due to the absence of the spontaneous shape deformation, but it can occur which leads to the vortex formation if we add an additional quartic potential. Moreover, the spontaneous shape deformation and consequently the formation of a vortex in an attractive system depends on the strengths of the two-body interaction and the quartic potential.