With an increasing demand for hydraulic pumps in the era of Industry 3.0 and 4.0 due to their higher performance, they are getting compact and thus more vulnerable to leakage due to tight tolerances. Thus, it is imperative to diagnose these faults at the early stage. In this study, an attempt has been made to diagnose the internal leakage of hydraulic pumps using an Exhaustive Feature Selection (EFS) with Random Forest (RF) classifier. The electrical power signal (unbalanced towards healthy condition) of the electric motor driving the pump has been used to diagnose the two different severities of internal leakage. Firstly, four statistical features namely skewness, kurtosis, shape indicator and impulse indicators are extracted. These four features are combined in all possible ways using EFS to get 15 different combinations. These 15 features are then used to train the random forest classifier and the accuracy and the standard deviation is evaluated for every combination. The methodology successfully determines the number of features best suited for the classification and accuracy corresponding to it. Secondly, the unbalanced dataset has been balanced using random under-sampling of healthy signals and the same methodology is used to ascertain the effect of balanced dataset classification using random forest. The study shows that a balanced dataset can be diagnosed more efficiently (89.21%) as compared to an unbalanced dataset (87.94%) using this methodology at the cost of loss of some information during under-sampling. In addition, another resampling method, Tomek Link has been employed. The test accuracy of trained random forest model is found to be highest among all (94.44%). It also decreases the false positive and false negative prediction for minority classes. Thus, this study provides an insight into the extensive combination of features and their effect on the pump leakage diagnosis and is highly useful when a limited amount of faulty data is available.