Reinforcement learning in batch processes

Conventional methods for batch chemical process optimisation and control depend on having both perfect process models and measurements available. Here, to avoid this, we apply a novel methodology centred on reinforcement learning (RL) whereby, unlike most forms of machine learning, an autonomous agent is not instructed on how to act by example but instead learns directly by trying control actions and seeking for those giving maximum reward. A central notion is the performance or value function that, in a given current state, signifies the contribution a specific action will make towards maximising the final performance or reward over an entire batch. For batch-to-batch, incremental learning and control, the initially unknown value function is here represented using wire fitting and a neural network. This is a simple yet powerful means of simultaneously learning and fitting the value function. The performance achieved in each completed batch can be propagated from the end point back through the intermediate states. With echoes of dynamic programming, this allows calculation of Bellman=s errors which can be minimised in neural network fitting. The higher level optimisation and control problem in batch processing thus fits neatly into this framework and some results of a case study illustrate the potential of the approach.