Big Data for Modern Industry: Challenges and Trends

We are living in an era of data deluge and as a result, the term big data is appearing in many contexts, from meteorology, genomics, complex physics simulations, biological and environmental research, finance and business to healthcare. One interesting example is that a press release of SAP AG, dated 11 June 2014 reported, SAP and the German Football Association turn big data into smart decisions to improve player performance at theWorld Cup in Brazil. An International Data Corporation (IDC) report [1] predicts that from 2005 to 2020, the digital universe will grow by a factor of 300, from 130 Exabyte to 40 000 Exabyte and that from now until 2020 will about double every two years. As the name implies, big data literally means large collections of data sets containing abundant information. However, it has some special characteristics that distinguish it from very large data or massive data that are simply enormous collections of simple-format records, typically equivalent to enormous spreadsheets. Big data, being generally unstructured and heterogeneous, is extremely complex to deal with via traditional approaches, and requires real-time or almost real-time analysis. A short definition can therefore be that big data refers to data sets whose size is beyond the ability of typical database software tools to capture, store, manage, and analyze. For a thorough discussion on various aspects of big data and the challenges it presents, together with some potential research directions, the reader is referred to [2]. © 1963-2012 IEEE.