LOCO-EPI: Leave-one-chromosome-out (LOCO) as a benchmarking paradigm for deep learning based prediction of enhancer-promoter interactions

In mammalian and vertebrate genomes, the promoter regions of the gene and their distal enhancers may be located millions of base-pairs from each other, while a promoter may not interact with the closest enhancer. Since base-pair proximity is not a good indicator of these interactions, there is a significant body of work to develop methods for understanding Enhancer-Promoter Interactions (EPI) from genetic and epigenomic marks. Over the last decade, several machine learning and deep learning methods have reported increasingly higher accuracies for predicting EPI. Typically, these approaches perform analysis by randomly splitting the dataset of Enhancer-Promoter (EP) pairs into training and testing subsets followed by model training. However, the aforementioned random splitting inadvertently causes information leakage by assigning EP pairs from the same genomic region to both testing and training sets. As a result, it has been pointed out in the literature that the performance of EPI prediction algorithms is overestimated because of genomic region overlap among the training and testing parts of the data. Building on that, in this paper we propose to use a more thorough training and testing paradigm i.e., Leave-one-chromosome-out (LOCO) cross-validation for EPI prediction. LOCO has been used in other bioinformatics contexts and ensures that there is no genomic overlap between training and testing sets enabling more fair estimation of performance. We demonstrate that a deep learning algorithm which gives higher accuracies when trained and tested on random-splitting setting, drops drastically in performance under LOCO setting, showing overestimation of performance in previous literature. We also propose a novel hybrid multi-branch neural network architecture for EPI prediction. In particular, our architecture has one branch consisting of a deep neural network, while the other branch extracts traditional k-mer features derived from the nucleotide sequence. The two branches are later merged and the neural network is trained jointly to force the network to learn feature representations which are already not covered by k-mer features. We show that the hybrid architecture performs significantly better in a realistic and fair LOCO testing paradigm, demonstrating it can learn more general aspects of EP interactions instead of overfitting to genomic regions. Through this paper we are also releasing the LOCO splitting-based EPI dataset to encourage other research groups to benchmark their EPI algorithms using a consistent LOCO paradigm. Research data is available in this public repository: https://github.com/malikmtahir/EPI