A Spark-based parallel genetic algorithm for Bayesian network structure learning

The Bayesian network structure learning (BNSL) algorithm based on genetic algorithm (GA) has the problem of long search time and being prone to falling into local optima. When the sampling data is large, the single machine BNSL algorithm cannot obtain the BN structure within a limited time. To address this issue, this paper proposes a parallel BNSL algorithm based on the Spark framework with GA (PGA-BN). The three main stages of the proposed PGABN are population initialisation, BIC score calculation, and evolution operators, which are all designed in parallel on each partition to accelerate based on Spark. The experiments are studied on two typical BN datasets with different sample sizes to evaluate the parallel performance of the PGA-BN algorithm. Experimental results showed that the PGA-BN is significantly faster than its single-machine version with the satisfied accuracy.