An Efficient Distributed Algorithm for Big Data Processing

This paper introduces an efficient distributed data analysis framework for big data which comprises data processing at the data collecting nodes and the central server end as opposed to the existing framework that only comprises data processing at the central server end. As data are being processed at the data collecting end in the proposed framework, the amount of data is reduced to be processed at the server side by the commodity computers. The proposed distributed algorithm works both in low-powered nodes such as sensors and high-speed commodity computers and also performs sequential and parallel processing based on the amount of data received at the central server. Simulation results demonstrate that the proposed distributed algorithm outperforms traditional distributed algorithms in terms of the size of data to be processed at the central server and data processing time.