EXACT: How to train your accuracy

Classification tasks are typically evaluated based on accuracy. However, due to the discontinuous nature of accuracy, it cannot be directly optimized using gradient-based methods. The conventional approach involves minimizing surrogate losses such as cross-entropy or hinge loss, which may result in suboptimal performance. In this paper, we introduce a novel optimization technique that incorporates stochasticity into the model's output and focuses on optimizing the expected accuracy, defined as the accuracy of the stochastic model. Comprehensive experimental evaluations demonstrate that our proposed optimization method significantly enhances performance across various classification tasks, including SVHN, CIFAR-10, CIFAR-100, and ImageNet.