Proximal Policy Optimization-Based Optimization of Microwave Planar Resonators

Microwave planar resonators have been widely used in the design of microwave and radio frequency (RF) circuits, and their geometrical variation plays a crucial role in determining the performance. An automatic optimization method for resonator structures is highly desirable. In this article, microwave planar resonator is abstracted into different models, and the transmission characteristics obtained from full-wave electromagnetic simulation are used as feedback rewards. The proximal policy optimization (PPO) framework from deep reinforcement learning (DRL) is employed, and multiple environments and action modes are proposed. The optimization is performed for both nonring and ring resonators. The developed optimization algorithm achieves a significant improvement in the performance compared to initial resonators. Specifically, using a chessboard-like optimization method, the negative group delay (NGD) of the microwave NGD component increased by 16%, and the return loss decreased by 11%. The sensitivity of the dielectric sensor improved by 16% when measuring deionized water. Additionally, by employing a path optimization method, the sensitivity of the microwave microfluidic sensor significantly increased by 62%. The experimental results demonstrate that the developed approach based on DRL can effectively explore the variability of resonator structures, and it has strong capability and generality.