Investigation on the Effect of L1 an L2 Regularization on Image Features Extracted using Restricted Boltzmann Machine

Regularization plays an important role in fine tuning the predictor design. In this research, we have investigated how L1 and L2 regularizations affect the image features. We have shown here the contrasting effect of 'L1' and 'L2' regularizations on the extracted features of images using Restricted Boltzmann Machine, an energy-based stochastic graphical technique for Classification. Through results, we show that 'L2' regularization produces feature with global receptive fields while 'L1' regularization produces feature with highly local receptive fields. Our findings have been validated with extensive simulation results and analysis on three datasets namely MNIST, CIFAR10 and custom AmigoBot images dataset collected in our lab. We conclude that L1 produces features which are spatially localized whereas L2 regularization produces features with higher spatial variance. These findings will be useful for deciding what kind of dataset requires what kind of regularization.