Comparing the comprehensibility of requirements models: An experiment replication

Context: There are several requirements modeling approaches with differences, for instance, in perspective, abstraction levels, modeling focus, and representation. Therefore, comparing and selecting a requirements modeling approach can be a difficult task. An important requirement for an approach is model comprehensibility. Objective: This paper replicates the experiment executed by Hadar et al. (2013) to compare the comprehensibility of Tropos requirements models and Use case models, along with the effort to comprehend the model, and the productivity. Method This replication varies the operationalization, protocol, population, and experimenters, addressing some future works proposed by the original experiment. Only one application domain of the original experiment was considered, but the same questionnaire was used to evaluate model comprehensibility. In addition to the two models of the original experiment, we also considered another Use case template. Results: Differently from the original experiment, the results of this replication indicate no difference in model comprehensibility and effort between Tropos and the two Use case models considered. As in the original experiment, the results indicate no difference in productivity. Conclusions: Differences in the experiment setting may explain the different results from the original experiment. Yet, it may be difficult to compare requirements approaches with complementary strengths and limitations as the requirements models must have equivalent content and complexity.