A/B Testing in E-commerce Sales Processes

E-commerce has traditionally been a field where online controlled experiments, such as A/B testing, take place. Most of these experiments focus on evaluating the front-end of the application and specifically different visual aspects, e. g. creating variations of the layouts, fonts, colors of the site, etc. In this paper we want to experiment whether A/B testing can be used to evaluate e-commerce sales processes and to improve the resilience of these processes. To achieve this goal we developed a tool in collaboration with a company, called Sonician AB, which is focused on marketing automation. The tool has been designed to empower business owners with a virtual assistant able to help customers understanding their needs and making decisions while purchasing products or services. The tool has been evaluated within the company and instantiated in two different business flow cases. Two experiments under real-life conditions show promising results. The paper concludes with lessons learned and a set of guidelines designed to help companies with interest of conducting similar experiments.