PBIL for Optimizing Hyperparameters of Convolutional Neural Networks and STL Decomposition

The optimization of hyperparameters in Deep Neural Networks is a critical task for the final performance, but it involves a high amount of subjective decisions based on previous researchers' expertise. This paper presents the implementation of Population-based Incremental Learning for the automatic optimization of hyperparameters in Deep Learning architectures. Namely, the proposed architecture is a combination of preprocessing the time series input with Seasonal Decomposition of Time Series by Loess, a classical method for decomposing time series, and forecasting with Convolutional Neural Networks. In the past, this combination has produced promising results, but penalized by an incremental number of parameters. The proposed architecture is applied to the prediction of the Rn-222 level at the Canfranc Underground Laboratory (Spain). By predicting the low-level periods of Rn-222, the potential contamination during the maintenance operations in the experiments hosted in the laboratory could be minimized. In this paper, it is shown that Population-based Incremental Learning can be used for the choice of optimized hyperparameters in Deep Learning architectures with a reasonable computational cost.