Higgs-boson-pair production H(→b(b)over-bar)H(→γγ) from gluon fusion at the HL-LHC and HL-100 TeV hadron collider

We perform the most up-to-date comprehensive signal -background analysis for Higgs -pair production in HH -> b (b) over bar gamma gamma channel at the HL-LHC and HL -100 TeV hadron collider, with the goal of probing the self-coupling.i3H of the Higgs boson which is normalized to its standard -model value of 1. We simulate all the standard -model signal and background processes and emphasize that the qgH(->gamma gamma) background has been overlooked in previous studies. We find that, even for the most promising channel HH -> b (b) over bar gamma gamma at the HL-LHC with a luminosity of 3000 fb(-1), the significance is still not high enough to establish the Higgs self-coupling at the standard -model (SM) value. Instead, we can only constrain the self-coupling to -1.0 < lambda(3H) < 7.6 at 95% confidence level after considering the uncertainties associated with the topYukawa coupling and the estimation of backgrounds. Here we also extend the study to the HL -100 TeV hadron collider. With a luminosity of 3 ab-1, we find there exists a bulk region of 2.6 less than or similar to lambda(3H) less than or similar to 4.8 in which one cannot pin down the trilinear coupling. Otherwise, one can measure the coupling with a high precision. At the SM value, for example, we show that the coupling can be measured with about 20% accuracy. While assuming 30 ab(-1), the bulk region reduces to 3.1 less than or similar to lambda(3H) less than or similar to 4.3, and the trilinear coupling can be measured with about 7% accuracy at the SM value.