Secure Transmit Antenna Selection Protocol for MIMO NOMA Networks Over Nakagami-m Channels

In this paper, we consider a multi-input multioutput (MIMO) nonorthogonal multiple access (NOMA) network consisting of one source and two legitimate users (LUs), so-called near and far users according to their distances to the source, and one passive eavesdropper, over Nakagami-m fading channels. Specifically, we investigate the cases where the signals of the far user might or might not be successfully decoded at the eavesdropper and the near user. Thus, we aim to design a transmit antenna selection (TAS) secure communication protocol for the network; where, two TAS solutions, namely Solutions I and II, are proposed. Specifically, Solutions I and II focus on maximizing the received signal power between the source and the near user, and between the source and the far user, respectively. Accordingly, exact and asymptotic closed-form expressions for the secrecy outage probability of the LUs and the overall system are derived. Our analytical results corroborated by the Monte Carlo simulation indicate that the secrecy performance could be significantly improved by properly selecting the power allocation coefficients and increasing the number of antennas at the source and the LUs. Interestingly, solution II is shown to provide a better overall secrecy performance over solution I.