Rare-event probability estimation with conditional Monte Carlo

Estimation of rare-event probabilities in high-dimensional settings via importance sampling is a difficult problem due to the degeneracy of the likelihood ratio. In fact, it is generally recommended that Monte Carlo estimators involving likelihood ratios should not be used in such settings. In view of this, we develop efficient algorithms based on conditional Monte Carlo to estimate rare-event probabilities in situations where the degeneracy problem is expected to be severe. By utilizing an asymptotic description of how the rare event occurs, we derive algorithms that involve generating random variables only from the nominal distributions, thus avoiding any likelihood ratio. We consider two settings that occur frequently in applied probability: systems involving bottleneck elements and models involving heavy-tailed random variables. We first consider the problem of estimating ℙ(X1+⋅⋅⋅+Xn>γ), where X1,…,Xn are independent but not identically distributed (ind) heavy-tailed random variables. Guided by insights obtained from this model, we then study a variety of more general settings. Specifically, we consider a complex bridge network and a generalization of the widely popular normal copula model used in managing portfolio credit risk, both of which involve hundreds of random variables. We show that the same conditioning idea, guided by an asymptotic description of the way in which the rare event happens, can be used to derive estimators that outperform existing ones.