Robust Joint Hybrid Transceiver Design for Millimeter Wave Full-Duplex MIMO Relay Systems

The joint design of hybrid beamforming matrices is conceived for multiuser mm-wave full-duplex (FD) multiple-input multiple-output (MIMO) relay-aided systems in the presence of realistic channel state information (CSI) errors. Specifically, considering a probabilistic CSI error model, we maximize the system's worst-case sum rate by jointly optimizing the base station's (BS's) analog and digital beamforming matrices, plus the analog receive and transmit beamforming matrices of the relay station (RS) as well as its digital amplify-and-forward beamforming matrix under practical constraints. Explicitly, the transmit power constraints of the BS and RS, the residual self-interference power constraint of the RS, the per-user quality of service constraints, and the unit-modulus constraints on the analog beamforming matrix elements are all taken into account. Since the resultant optimization problem is very challenging due to its highly nonlinear objective function and nonconvex coupling constraints, we first transform it into a more tractable form. We then develop a novel joint optimization algorithm based on the penalty dual decomposition (PDD) technique to solve the resultant problem. The proposed PDD-based algorithm performs double-loop iterations: the inner loop updates the optimization variables in a block coordinate descent fashion, while the outer loop adjusts the Lagrange multipliers and penalty parameter, hence ensuring convergence to the set of stationary solutions of the original problem. Our simulations show that the mm-wave FD hybrid MIMO relay systems relying on our new algorithm significantly outperform both their non-robust FD and conventional half-duplex counterparts.