Error analysis of TAS-OSTBC assisted downlink NOMA system over generalized η-μ$$ \eta -\mu $$ fading Channel

Incorporation of multiple antennas in downlink non-orthogonal multiple access (NOMA) system helps in achieving better performance by exploiting different diversity schemes. To this end, we propose a general framework of transmit antenna selection (TAS)-assisted orthogonal space-time block code (OSTBC) transmission for multiple NOMA users. We derive exact expressions of probability density function of the TAS-OSTBC processed signal-to-noise ratio (SNR) at the output of maximum-ratio combiner of the NOMA users. Using well-known moment generating function approach, we evaluate the error performance of TAS-OSTBC assisted NOMA users over generalized eta-mu fading channel in presence of perfect and imperfect successive interference cancelation. We investigate the proposed system in different multi-antenna scenarios and accordingly observe better error performance of the users by increasing the number of transmit and receive antennas. We perform their asymptotic analysis leading to full-diversity order at high SNR. For numerical evaluation, we truncate the infinite series in derived results and examine the upper bound of truncation error. We evaluate nth moment of the output SNR and obtain channel quality estimation index as a performance measure of the proposed NOMA system. Moreover, we investigate the influence of power coefficients and fading parameters, eta and mu, on the error performance of TAS-OSTBC-assisted NOMA users. We also compare the derived error performances with the existing schemes, wherein we observe that the proposed system shows remarkable improvement in the bit error rate (BER) of the NOMA users. For validation, we execute simulations which closely match with the derived analytical results.