5G core frequency band, mmWave, is about ten times wider than the existing commercial frequency band, enabling various services to be created. However, due to the characteristics of the mmWave frequency, it has various limitations. In the mobile environment, the misaligned beam problem, in which the SNR is degraded because the beam align between the sender and the receiver does not match, is one of the biggest problems to be solved. In this paper, we propose a adaptive beam management scheme based on deep-learning to solve misaligned beam problem. In the proposed scheme, 5G base-station (gNB) learns the mobility information, SNR, and current beam information of the associated user equipment (UE) by the deep-learning agent, and predicts whether the beam is aligned or not. From the prediction result, gNB and UE perform beam hands-off in advance before loss of connectivity.