In the multi-access edge computing (MEC), task offloading through device-to-device (D2D) communication can improve the performance of edge computing by utilizing the computational resources of nearby mobile devices (MDs). However, adapting to the time-varying wireless environment and efficiently and quickly allocating tasks to MEC and other MDs to minimize the energy consumption of MDs is a challenge. First, we constructed a multi-device collaborative task offloading framework, modeling the collaborative task offloading decision problem as a graph state transition problem and utilizing a graph neural network (GNN) to fully explore the potential relationships between MDs and MEC. Then, we proposed a collaborative task offloading algorithm based on graph reinforcement learning and introduced a penalty mechanism that imposes penalties when the tasks of MDs exceed their deadlines. Simulation results show that, compared with other benchmark algorithms, this algorithm reduces energy consumption by approximately 20%, achieves higher task completion rates, and provides a more balanced load distribution.