Near-Optimal Algorithms for Private Online Optimization in the Realizable Regime

We consider online learning problems in the realizable setting, where there is a zero-loss solution, and propose new Differentially Private (DP) algorithms that obtain near-optimal regret bounds. For the problem of online prediction from experts, we design new algorithms that obtain near-optimal regret (O) over tilde(epsilon(-1) log(1.5) d) where d is the number of experts. This significantly improves over the best existing regret bounds for the DP non-realizable setting which are (O) over tilde(epsilon(-1) min {d, T-1/3 log d}). We also develop an adaptive algorithm for the small-loss setting with regret O(L-star log d + epsilon(-1) log(1.5) d) where L-star is the total loss of the best expert. Additionally, we consider DP online convex optimization in the realizable setting and propose an algorithm with near-optimal regret (O) over tilde(epsilon(-1) d(1.5)), as well as an algorithm for the smooth case with regret (O) over tilde(epsilon(-2/3) (dT)(1/3)), both significantly improving over existing bounds in the non-realizable regime.