Deep Learning-Based MIMO-NOMA With Imperfect SIC Decoding

Nonorthogonal multiple access (NOMA) and multiple-input multiple-output (MIMO) are two key enablers for 5G systems. In this article, considering the practical issue that successive interference cancellation (SIC) decoding is imperfect in the real-world NOMA system, we propose a novel scheme for the downlink of the MIMO-NOMA system based on deep learning. In this scheme, both precoding and SIC decoding of the MIMO-NOMA system are jointly optimized (or learned) in the sense of minimizing total mean square error of the users' signals. To this end, we construct the precoder and SIC decoders using deep neural networks such that the transmitted signals intended to multiple users can be properly precoded at the transmitter based on the superposition coding technique and the received signals are accurately decodable at the users by the SIC decoding. Numerical results demonstrate the effectiveness and superior performance of the proposed scheme.