Ionic conduction plays a critical role in the process of electrode reactions and the charge transfer kinetics in a rechargeable battery. Covalent organic frameworks (COFs) have emerged as an exciting new class of ionic conductors, and have made great progress in terms of their application in rechargeable batteries. The unique features of COFs, such as well-defined directional channels, functional diversity, and structural robustness, endow COF-based conductors with a low ionic diffusion energy barrier and excellent temperature tolerance, which are much superior to the classic inorganic or polymer conductors. Here, a comprehensive analysis and summary of the unique ion-conducting behavior of COF-based conductors are presented, and the design principles for ion-conducting COFs are emphasized. Moreover, a systematic overview of the recent progress in the development of ion-conducting COFs serving as electrodes, separators, solid electrolytes, and artificial interphase materials for diverse battery applications, such as metal-ion batteries, lithium metal batteries, lithium-sulfur batteries, lithium-CO2 batteries, zinc-air batteries, etc., is proposed. This review is expected to provide theoretical guidance for design of novel kinds of ionic conductor bearing intrinsic framework structures and to boost further research enthusiasm for ion-conducting COFs in rechargeable batteries.