Shape Penalized Decision Forests for Imbalanced Data Classification

Class imbalance poses a critical challenge in binary classification problems, particularly when rare but significant events are underrepresented in the training set. While traditional machine learning models and modern deep learning techniques struggle with such imbalances, decision trees and random forests combined with data sampling strategies have shown effectiveness, especially for tabular datasets. However, undersampling and oversampling approaches often introduce complexity and risk the loss of valuable information. This paper introduces Shape Penalized Decision Forests, a novel classifier tailored for imbalanced binary classification. Our method integrates a penalty on the surface-to-volume ratio of decision sets within decision tree construction, thereby inherently addressing class imbalance without additional sampling. The proposed approach enhances predictive performance and generalization by leveraging ensemble learning strategies such as bagging and adaptive boosting. We evaluate the method on twenty benchmark tabular imbalanced datasets, spanning diverse sample sizes and imbalance ratios, and demonstrate its superiority over several state-of-the-art data-level and algorithmic-level methods. Furthermore, simulated datasets with visually interpretable structures showcase the model's generalization capacity. Statistical significance tests validate the robustness of our approach. Finally, we provide a Python package, 'imbalanced-spdf', offering an accessible implementation for practitioners and researchers.