Appearing as the next generation of the grid system, smart grid has extended to an intelligent system that features numerous equipment access and massive data exchange. This inevitably incurs rampant cyber and physical attacks, and authentication is an optimal solution. However, most authentication protocols recently are vulnerable to physical attacks, and are not suitable for the end-edge-cloud architecture of smart grid. In this paper, we propose a batch Authentication and Key Agreement (AKA) protocol with a binding scheme for hierarchical smart grid. The intrinsic Physically Unclonable Function (PUF) is employed to guard both meters and gateways deployed in an open environment. Mutual end-to-end AKA is established between meters and the server, which provides integrated security for both wired bus and wireless Narrowband Internet of Things (NB-IoT). The intermediate gateways are endowed with batch authentication and access control. To prove the security, Tamarin is employed to give a formal verification under the extended Dolev-Yao (DY) model and honest-but-curious gateway. By using lightweight primitives, our protocol is deployed on resource-constrained MSP430FR5969. Performance in terms of communication, signaling, and storage overhead is also analyzed to demonstrate the superiority to others.