Transceiver Design for AF MIMO Two-Way Relay Systems with Imperfect Channel Estimation

We address the transceiver design problem for amplify-and-forward (AF) MIMO two-way relay systems with imperfect channel estimation. Since only estimated channel state information (CSI) is available, the self-interference (SI) cannot be completely canceled at destination nodes. With both channel estimation errors and residual self-interference considered, we propose two robust schemes to minimize the average sum mean squared error (MSE). In the iterative scheme, the relay precoder and destination receivers are alternately optimized until convergence. In the constrained structure scheme, a specific structure is imposed on the relay precoder to reduce complexity. By solving a relaxed optimization problem, we derive a closed-form solution for the constrained structure precoder. Simulation results show that the proposed iterative scheme provides better robustness against channel uncertainties than the non-robust iterative scheme, and the constrained structure scheme achieves close performance with the proposed iterative scheme.