Data Driven Development: Challenges in Online, Embedded and On-Premise Software

For more than a decade, data driven development has attracted attention as one of the most powerful means to improve effectiveness and ensure value delivery to customers. In online companies, controlled experimentation is the primary technique to measure how customers respond to variants of deployed software. In B2B companies, an interest for data driven development is rapidly emerging and experiments are run on selected instances of the system or as comparisons of previously computed data to ensure quality, improve configurations and explore new value propositions. Although the adoption of data driven development is challenging in general, it is especially so for embedded systems companies and for companies developing on-premise software solutions. Due to complex systems with hardware dependencies, safetycritical functionality and strict regulations, these companies have longer development cycles, less frequent deployments and limited access to data. In this paper, and based on multi-case study research, we explore the specific challenges that embedded systems companies and companies developing onpremise solutions experience when adopting data driven development practices. The contribution of the paper is two-fold. First, we provide empirical evidence in which we identify the key challenges that embedded systems and on-premise software solutions companies experience as they evolve through the process of adopting data driven development practices. Second, we define the key focus areas that these companies need to address for evolving their data driven development adoption process.