Carbon fiber reinforced polymer (CFRP) is prone to surface damage such as delamination, uncut fibers and tear during the process of hole-making in aerospace field. Helical milling greatly improves the exit damage of CFRP holes, but the entrance damage will further deteriorate. To further diminish hole entrance damage and enhance hole machining quality, this paper proposed a longitudinal torsional ultrasonic assisted helical milling method to investigate the formation mechanism of hole entrance damage based on a cutting fracture mechanism with a fiber angle of 0 degrees to 180 degrees. The differences of hole entrance damage between longitudinal torsional ultrasonic helical milling (LTUHM) and traditional helical milling (THM) were analysed by a series of comparative experiments. The results showed that longitudinal torsional ultrasonic machining significantly reduced the damage of CFRP holes compared to THM. The delamination damage factor and uncut fibers factor of the hole entrance are reduced to 24.92% and 20.28%, respectively, and the fiber fracture surface is flatter under LTUHM. The study provides a production guide for efficient hole-making of CFRP.