The perturbation problem of symmetries for constrained Hamiltonian systems under the action of small excitation is studied. Firstly, the canonical equations of nonconservative constrained Hamiltonian systems are established, and the symmetries and exact invariants of the systems in the extended phase space are studied. Secondly, according to the concept of high-order adiabatic invariants of dynamical systems, this paper gives the form of adiabatic invariants and the conditions for their existence and establishes the relationship between adiabatic invariants and symmetrical transformations. Finally, an example is given to illustrate the application of the results.