Within the concept of physical human-robot interaction (pHRI), the paramount criterion is the safety of the human operator interacting with a high-degree-of-freedom (DoF) robot. Consequently, there is a substantial demand for a robust control scheme to establish safe pHRI and stabilize nonlinear, high DoF systems. In this article, an adaptive decentralized control strategy is designed to accomplish the abovementioned objectives. A human upper limb model and an exoskeleton model are decentralized and augmented at the subsystem level to enable decentralized control action design. Human exogenous torque (HET), which can resist exoskeleton motion, is estimated using radial basis function neural networks (RBFNNs). Estimating human upper limb and robot rigid body parameters, as well as HET, makes the controller adaptable to different operators, ensuring their physical safety during the interaction. To guarantee both safe operation and stability, the barrier Lyapunov function (BLF) is utilized to adjust the control law. This study also considers unknown actuator uncertainties and constraints to ensure a smooth and secure pHRI. In addition, it is shown that incorporating RBFNNs and BLF into the original virtual decomposition control (VDC) improved its performance. The asymptotic stability of the entire system is established through the concept of virtual stability and virtual power flows (VPFs) under the proposed robust controller. Experimental results are presented and compared with those obtained using proportional-derivative (PD) and proportional-integral-derivative (PID) controllers to showcase the robustness and superior performance of the designed controller, particularly in controlling the last two joints of the robot.