Intelligent reflecting surface (IRS) is considered as a potential candidate to reconfigure the propagation environment, avoiding blockage and enhancing spectral efficiency. Due to its passive feature, IRS-aided wireless systems are more in line with the vision of the energy-efficient communications. In this article, a phase modulation array (PMA) based hybrid precoding framework in IRS-aided system is proposed, in which the PMA-based analog precoder is employed to enhance energy efficiency. To design hybrid precoding and determine phase shift of IRS, a three-matrix-variable optimization problem is formulated, which cannot be directly dealt with for both single-user and multi-user systems. Thus, it is divided into two individual sub-problems. Then, a maximizing array gain (MAG) based algorithm is proposed to seek the IRS reflection matrix. Furthermore, in order to reduce the computational complexity, a deep neural network (DNN) based algorithm is also proposed to acquire better IRS design. In addition, to deal with the IRS aided multi-user system, an equivalent channel gain maximization method and another DNN-based algorithm are also designed to obtain the IRS reflection matrix. Simulation results verify that, the proposed joint hybrid precoder-IRS design scheme outperforms other existing schemes both in spectral efficiency and energy efficiency.