Merging compact binaries near a rotating supermassive black hole: Eccentricity excitation due to apsidal precession resonance

We study the dynamics of merging compact binaries near a rotating supermassive black hole (SMBH) in a hierarchical triple configuration. We include various general relativistic effects that couple the inner orbit, the outer orbit and the spin of the SMBH. During the orbital decay due to gravitational radiation, the inner binary can encounter an "apsidal precession resonance" and experience eccentricity excitation. This resonance occurs when the apsidal precession rate of the inner binary matches that of the outer binary, with the precessions driven by both Newtonian interactions and various post-Newtonian effects. The eccentricity excitation requires the outer orbit to have a finite eccentricity, and is most effective for triples with small mutual inclinations, in contrast to the well-studied Lidov-Kozai effect. The resonance and the associated eccentricity growth may occur while the binary emits gravitational waves in the low-frequency band, and may be detectable by future space-based gravitational wave detectors.