Energy harvesting (EH)-enabled cellular Internet of Things (IoT) is a promising solution to handle the charging and accessing of massive IoT nodes. However, limited by the high-frequency band of future 5G, the radius of the small base station (SBS) is reduced, hence greatly increasing the cost of the network operators (NOs). In this article, we consider the joint cell association, cell sleeping (CS), and incentive decision problem for EH-aided cellular IoT with full-duplex (FD) self-backhauls. We formulate a Stackelberg game to investigate the coordination between the utilities of NO and energy transmitters (ETs), where both the features of FD self-backhauls and CS are introduced to reduce the expense of NO. We then propose an alternative direction algorithm to solve the equilibrium of the game efficiently, where the relationship of the formulated constraints and variables are utilized to transform the original problem into two subproblems. We propose a two-level Lagrangian relaxation to solve the first subproblem, while the other is proved to be convex and solved by an efficient iteration. Simulation results demonstrate the benefits of our algorithm in utility improvement and expense reduction. Moveover, it shows that our algorithm can obtain high efficiency by adjusting the tradeoff between the number of active SBS and transmitting power of ETs according to the network deployment.