With the rapid development of low earth-orbit (LEO) satellites, the ultra-dense LEO satellite network (UDLSN) has become a promising solution to provide Internet services for civil aviation due to its wide coverage, exceptional flexibility and high reliability. To guarantee the continuity and high-quality Internet services, satellite-aircraft handover is inevitable because of the high mobility of aircraft and satellites. However, the stringent passenger communication requirements and the considerable increase in the scale of candidate satellites pose huge challenges in satellite-aircraft handover in UDLSNs. In this paper, we investigate multiple civil aircraft handover issue including satellite handover, subchannel allocation, and power selection in the UDLSN and consider different levels of users & acirc; rate satisfaction, power overhead, and handover overhead as handover criteria for performance measure. The optimization problem is formulated to maximize the handover satisfaction of all civil aircraft. Particularly, the formulated handover problem is represented as a local cooperation game, where each aircraft determines the handover action and the power level by cooperating with other interference aircraft. We prove that the proposed game has at least one Nash Equilibrium (NE) solution in which no aircraft changes its handover strategy. The NE solution in the proposed game is further proven to locally or globally maximize the optimization objective and we then design the parallel handover strategy update (PHSU) algorithm to find the NE solution. Simulation results based on trajectories of real civil aircraft demonstrate that, the designed algorithm is robust, can effectively meet diverse user requirements, and reduce both handover frequency and delays.