A modification of logistic regression with imbalanced data: F-measure-oriented Lasso-logistic regression

Logistic regression (LR) is one of the most popular classifiers. However, LR cannot perform effectively on imbalanced data. There are two approaches to imbalanced data for LR, including resampling techniques and modifications to the log-likelihood function. These approaches improve performance measures of LR in some cases, but their effectiveness is not robust in general. In this paper, we propose a classifier called F-measure-oriented Lasso-Logistic Regression (F-LLR) to deal with imbalanced data. The base learner of F-LLR is Lasso-Logistic regression (LLR) which imposes the prior on the magnitude of parameters by a hyper-parameter a.. The optimal a. is determined by an adjustment of the cross-validation procedure which aims for the highest F-measure instead of the highest accuracy. F-LLR addresses imbalanced data by the combination of Under-sampling and Synthetic Minority Oversampling Technique (SMOTE) selectively based on the scores of the training data. The empirical study shows that F-LLR increases F-measure and KS as compared with LLR and the traditional balanced methods, such as the resampling techniques (Random Under-sampling, Random Over-sampling, and SMOTE) and the modifications to log-likelihood function (Ridge and Weighted likelihood estimation).