HawkEye: A robust loss function for regression with bounded, smooth, insensitive zone characteristics

Support vector regression (SVR) encounters challenges when confronted with outliers and noise, primarily due to the limitations of the traditional E-insensitive loss function. To address this, bounded loss functions have gained traction for their robustness and improved generalization. More recent advancements, BLINEX and bounded least square loss, focus on smooth bounded loss functions that enable efficient gradient based optimization. However, these approaches lack an insensitive zone, which is crucial for mitigating deviations and noise. The challenge of designing a loss function that combines boundedness, smoothness, an insensitive zone remains unresolved in the current literature. To address this issue, we develop the HawkEye loss, a novel formulation that integrates boundedness, smoothness, and the presence of an insensitive This unique combination enhances the robustness and generalization capabilities of SVR models, particularly the presence of noise and outliers. Notably, the HawkEye loss is the first in SVR literature to simultaneously incorporate boundedness, smoothness, and an insensitive zone. Leveraging this breakthrough, we integrate the HawkEye loss into the least squares framework of SVR and yield a new robust and scalable termed HE-LSSVR. The optimization problem inherent to HE-LSSVR is addressed by harnessing the adaptive moment estimation (Adam) algorithm, known for its adaptive learning rate and efficacy in handling scale problems. To our knowledge, this is the first time Adam has been employed to solve an SVR problem. empirically validate the proposed HE-LSSVR model, we evaluate it on UCI, synthetic, time series, and age datasets. The experimental outcomes unequivocally reveal the superiority of the HE-LSSVR model terms of its remarkable generalization performance and its efficiency in training time. The code of the proposed model is publicly available at https://github.com/mtanveer1/HawkEye.