A Linear Fractional Semidefinite Relaxation Approach to Maximum-Likelihood Detection of Higher-Order QAM OSTBC in Unknown Channels

This paper considers the blind maximum-likelihood (ML) detection problem for orthogonal space-time block codes (OSTBCs) in multiple-input multiple-output flat-fading channels. While the blind ML detection problem for general space-time codes is difficult to solve, it has been shown that for OSTBCs with constant modulus constellations, the blind ML detection problem can be formulated as a discrete quadratic program, and then handled by a powerful convex approximation technique known as semidefinite relaxation (SDR). In this paper, we turn our attention to the case of higher order QAM OSTBCs. Due to the nonconstant modulus nature of higher order QAM signals, the blind ML detection problem turns out to be a discrete Rayleigh quotient maximization problem, and as a result the current SDR technique is no longer directly applicable. We propose a linear fractional SDR (LFSDR) approach to this problem. This approach first relaxes the higher order QAM blind ML detection problem into a quasi-convex problem, followed by a simple solution approximation procedure. In general, quasi-convex problems are computationally more complex to solve than convex problems, but we show that an optimum solution of our quasi-convex problem can be efficiently obtained by solving a convex semidefinite program. The approximation accuracy of the proposed approach relative to other possible relaxation approaches is also studied. Simulation results are presented to demonstrate that the proposed LFSDR-based blind ML detector outperforms some existing suboptimal detectors and can yield promising performance even with a small to moderate number of code blocks.