Prototyping and In-Depth Analysis of Big Data Benchmarking

Today's digital age has witnessed an explosion of data and information. This has resulted into changing the nature of data from being a medium of supporting transactions to becoming a transactional commodity itself. The consequential increase in the value of data has led to many innovations in both academic and industrial circles. The main focus remains on finding efficient ways to analyse data and derive meaningful results out of it. An efficient way of doing so is constructing benchmarks in order to effectively evaluate the performances of existing and upcoming data systems. A successful benchmark should cover all the major big data system application domains and there workloads. A prototype, outlining a small and diverse benchmark, which takes minimum time to cover a wide range of applications needs to be developed. In designing this prototype the four cornerstones of big data namely volume, veracity, velocity and variety should also be maintained. In addition to this the workload of a benchmark set should be carefully selected. It should represent a wide spectrum of application domains; have diversity of data characteristics and should not have any redundancy. Lastly, there should be a metric to evaluate the benchmarks so as to give them validity.