Technological Surveillance in Big Data Environments by using a MapReduce-based Method

For many years, the organizations had monitored the technological environments to anticipate changes with potentially positive or negative impacts on their business using technological surveillance process. However, the new Big Data scenarios turned the traditional tools and methods no longer sufficient. This paper proposes an automated technological surveillance method by using a map-reduce model to deal with Big Data scenarios divided into five processes: planning, collection, organization, intelligence, and communication. We implemented a system prototype to validate the proposed approach. It was developed in Python and Javascript, using ontologies for knowledge modeling, NoSql database to store and parallel processing of the publications. The system collected 2,918 publications, identified the monitored technologies, extracted the metadata, analyzed them, and generated charts for the stakeholders. In conclusion, the method demonstrated be feasible to automate the technology watch process in Big Data scenarious and dramatically reduced the workload involved when implemented by a system, offering a solid approach to automatically identify a set of technologies with increasing popularization in Web portals.