Indirect unitarity violation entangled with matter effects in reactor antineutrino oscillations

If finite but tiny masses of the three active neutrinos are generated via the canonical seesaw mechanism with three heavy sterile neutrinos, the 3 x3 Pontecorvo-Maki-Nakagawa-Sakata neutrino mixing matrix Vwill not be exactly unitary. This kind of indirect unitarity violation can be probed in a precision reactor antineutrino oscillation experiment, but it may be entangled with terrestrial matter effects as both of them are very small. We calculate the probability of (nu) over bar (e) -> (nu) over bar (e) oscillations in a good analytical approximation, and find that, besides the zero-distance effect, the effect of unitarity violation is always smaller than matter effects, and their entanglement does not appear until the next-to-leading-order oscillating terms are taken into account. Given a 20-kiloton JUNO-like liquid scintillator detector, we reaffirm that terrestrial matter effects should not be neglected but indirect unitarity violation makes no difference, and demonstrate that the experimental sensitivities to the neutrino mass ordering and a precision measurement of theta(12) and Delta(21) equivalent to m(2)(2) - m(1)(2) are robust. (C) 2018 The Author(s). Published by Elsevier B.V.