Efficient Sensory Data Transformation: A Big Data Approach

Big Data Analytics has immensely solved complex problems involving massive and complex data. Data filtering plays a vital role in helping the analysis processes to perform analytics with ease and in precise form. The data variety and veracity are some of the major problems that add enough complexity to data, creating the overall hindrance to an effective big data analytics process. The proposed work targets the data staging phase in a big data classification to tackle variety and veracity problems found in massive sensory data. The study adopts a novel approach for data storage, then designs and implements a simple algorithm to perform data transformation. The concept of cloud storage-bucket is used for effective storage and transformation of sensory data. Such an analytical approach is proven to retain the capability of an effective and faster data transformation. The algorithm performs conversion of unstructured data to semi-structured data in the first stage, then converts the semi-structured data to structured data in the second stage, and finally stores the resulting structured data into virtually localized distributed storage. The outcome of this study offers faster response time and higher data purity for data transformation process of data staging phase in any big data analytics application.