Machine Learning-Enabled Joint Codebook Design and Beam Selection

Recently, analog-digital hybrid precoding has emerged as a promising technique for mmWave (mmWave) multiuser multiple-input multiple-output (MU-MIMO) systems. However, it encounters two practical challenges in codebook design and beam selection. Traditional codebooks, once deployed, lack adaptability to the changing wireless landscape, while existing beam selection algorithms often fail to achieve optimal performance within reasonable complexity constraints. To address these issues, we propose a machine learning-enabled framework for joint codebook design and beam selection in mmWave MU-MIMO systems. Firstly, we design a learnable codebook using deep neural networks (NN). Then, we create a novel three-layer network architecture to enable the codebook's dynamic adaptation to changes in the wireless environment. Subsequently, the beam selection problem is formulated as a Markov decision process, effectively solved using a double deep Q-network algorithm. Then, the joint training of the above two NNs is developed to jointly optimize the codebook and beam selection. Simulation results demonstrate that the proposed approach significantly improves the system's sumrate compared to traditional methods.