Precoding Design for Two-Way MIMO Relaying with Antenna Selection

In the paper, joint source-and-relay optimization is proposed for two-way multiple-input multiple-output (MIMO) amplify-and-forward (AF) multi-relay systems with using a layered relay-and-antenna selection (LRAS) mechanism under power constraints in correlated channels. This minimum mean-squared error (MMSE) based optimization results in a highly nonlinear objective function such that an iterative algorithm is utilized to jointly optimize the source, relay, and receive matrices under power constraints. Simulation results show that the joint architecture of transceiver/relay precoders in conjunction with the LRAS strategy is capable of accomplishing superior performance in terms of MSE and bit-error-rate (BER) compared with other existing transceiver design algorithms.