This is the Python codes related to the following article: Yu Zhang, Muhammad Alrabeiah, and Ahmed Alkhateeb, “Reinforcement Learning of Beam Codebooks in Millimeter Wave and Terahertz MIMO Systems,” in IEEE Transactions on Communications, 2021.

Millimeter wave (mmWave) and terahertz MIMO systems rely on pre-defined beamforming codebooks for both initial access and data transmission. These pre-defined codebooks, however, are commonly not optimized for specific environments, user distributions, and/or possible hardware impairments. This leads to large codebook sizes with high beam training overhead which makes it hard for these systems to support highly mobile applications. To overcome these limitations, this paper develops a deep reinforcement learning framework that learns how to optimize the codebook beam patterns relying only on the receive power measurements. The developed model learns how to adapt the beam patterns based on the surrounding environment, user distribution, hardware impairments, and array geometry. Further, this approach does not require any knowledge about the channel, RF hardware, or user positions. To reduce the learning time, the proposed model designs a novel Wolpertinger-variant architecture that is capable of efficiently searching the large discrete action space. The proposed learning framework respects the RF hardware constraints such as the constant-modulus and quantized phase shifter constraints. Simulation results confirm the ability of the developed framework to learn near-optimal beam patterns for line-of-sight (LOS), non-LOS (NLOS), mixed LOS/NLOS scenarios and for arrays with hardware impairments without requiring any channel knowledge.

1. Download all the files of this repository.
2. Run main.py.
3. When main.py finishes, run read_beams.py.
4. Load beam_codebook.mat in Matlab.
5. Run plot_pattern(beams.') in Matlab Command Window, which will give Fig. 7(c) shown below as result.

If you have any problems with generating the figure, please contact Yu Zhang.

This code package is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License. If you in any way use this code for research that results in publications, please cite our original article:

Yu Zhang, Muhammad Alrabeiah, and Ahmed Alkhateeb, “Reinforcement Learning of Beam Codebooks in Millimeter Wave and Terahertz MIMO Systems,” in IEEE Transactions on Communications, 2021.