Professionals are not Superman: Failures beyond Motivation in Software Experiments

Background: Industry experiments are typically associated with higher external validity compared to academic experiments. However, when conducting industry experiments, dropouts and incomplete experimental tasks are quite common, which is unusual in academic experiments. To the best of our knowledge, this phenomenon has not been reported in the literature. Aim: Identify the circumstances that explain why some experimental subjects exhibit poor or null participation during experimental sessions. Method: An industry experiment with experienced programmers at the Universidad de las Fuerzas Armadas ESPE of Ecuador was performed. Several post hoc analyses of the experimental data revealed relationships that could explain the subjects' behavior. Results: A high percentage of older experienced programmers did not perform meaningful work in their task assignments, even though they were present during the entire experiment. Longer overall (i. e., not only programing) experience and poor knowledge of the programming language and integrated development environment have a negative influence in the degree of task completion as well. Conclusions: Several experienced professionals were found to live a two, mixed-factors reality: old age and technological lapse. This negatively influenced (to a greater or lesser extent, depending on the person) attitudes regarding performance of activities that differ from daily professional work.