Very large data sets within the range of megabytes to terabytes generated daily from checkpoint-and-restart processes are seen in today's scientific simulations. Reliability and durability are two important factors to build an archive storage system. Erasure code based object storage systems are becoming popular choices for archive storage systems due to cost-effective storage space saving schemes and higher fault-resilience capabilities. Both erasure code encoding and decoding procedures involve heavy array, matrix, and table-lookup compute intensive operations. Current solutions of the erasure coding process are based on single process approach which is not capable of processing very large data sets efficient and effectively. In this paper, we address the bottleneck problem of single process erasure encoding by leveraging task parallelism offered by multi-core computers. We add parallel processing capability to the erasure coding process. More specifically, we develop a parallel erasure coding software, called parEC. It explores the MPI run time parallel I/O environment and integrates data placement process for distributing encoded data blocks to destination storage devices. We evaluate the performance of parEC in terms of both encoding throughput and energy efficiency. We also compare the performance of two task scheduling algorithms for parEC. Our experimental results show parEC can significantly reduce the encoding time (i.e., by 74.06%-96.86%) and energy consumption (i.e., by 73.57%-96.86%), and Demand-based Workload Assignment (DBWA) algorithm can a high system utilization (i.e., 95.23%).
