When reformulated as a resource theory, thermodynamics can analyze system behaviors in the single-shot regime. In this, the work required to implement state transitions is bounded by alpha-R & eacute;nyi divergences and so differs in identifying efficient operations compared to stochastic thermodynamics. Thus, a detailed understanding of the difference between stochastic and resource-theoretic thermodynamics is needed. To this end, we explore reversibility in the single-shot regime, generalizing the two-level work reservoirs used there to multilevel work reservoirs. This achieves reversibility in any transition in the single-shot regime. Building on this, we systematically develop multilevel work reservoirs in the nondissipation regime with and without catalysts. The resource-theoretic results show that two-level work reservoirs undershoot Landauer's bound, misleadingly implying energy dissipation during computation. In contrast, we demonstrate that multilevel work reservoirs achieve Landauer's bound while producing arbitrarily low entropy.