Performance analysis of multicell massive MIMO THP with pilot contamination

Pilot contamination (PC) is a performance limiting factor in multicell massive multiple-input-multiple-output systems. The PC may cause the channel estimation of users in the desired cell to be polluted with the channels of users with the same set of pilots in the adjacent cells. In this paper, we propose a multicell Tomlinson-Harashima precoding (MC-THP) to mitigate PC. The MC-THP matrices are derived by the optimization function in which the errors seen by the users in the desired cell and the interference seen by the users in the neighboring cells are jointly minimized. In addition, the robust MC-THP design with imperfect channel state information is investigated based on the covariance matrices of the channel estimation errors of the desired and interfering channels. Finally, an approximate analytical closed-form of symbol error rate is obtained through the analysis of error covariance matrix in the desired cell. This is then used for power loading technique to improve the performance of the proposed MC-THP. The simulation results confirm the improvement of the proposed MC-THP concerning the conventional linear multicell minimum mean-square error (MC-MMSE) precoder.