Optimization of Fed-Batch Baker's Yeast Fermentation Using Deep Reinforcement Learning

Fermentation is widely used in chemical industries to produce valuable products. It consumes less energy and has a lesser environmental impact compared to conventional chemical processes. However, the inherent nonlinearity of the fermentation process and limited comprehension of its metabolic mechanisms present challenges for control and optimization, particularly in minimizing the formation of by-products. Reinforcement learning is a machine learning method where an agent learns through exploration and experience. By receiving feedback in the form of rewards, it computes an optimal policy which produces maximum cumulative reward. The integration of deep learning with reinforcement learning has further improved the efficiency of classical reinforcement learning, particularly in continuous control. This paper focuses on optimizing substrate feeding rate in a simulated fed-batch baker's yeast fermentation using deep reinforcement learning. Artificial neural network (ANN) was applied as the function approximator to estimate the state-action function for determining the substrate feeding rate within a large state space, which includes substrate concentration, yeast concentration, and the change in ethanol concentration. The deep reinforcement learning algorithm was formulated based on the optimization objective of maximizing yeast production while minimizing ethanol formation. The performance of the feeding strategy proposed using deep reinforcement learning was compared to a commonly used pre-determined exponential feeding profile. The results show that the proposed feeding strategy outperformed the exponential feeding strategy with the yeast yield increased by 25.66% with negligible ethanol production. In addition, the proposed algorithm exhibits effective handling of various initial conditions compared to the exponential feeding approach.