Task Completeness Assessments in the Evolution of Domain-Specific Modelling Languages

[Background] Domain-specific modelling languages (DSMLs) are tailored to particular application domains and are common in model-driven information system engineering. To support new modelling requirements, increase the maturity of the languages, and keep them relevant to their domain, DSMLs need to be evolved. [Aims] Since little is known regarding the complexity of the evolution process, in this paper, we investigate which incompletions are prevalent in each DSML evolution activity. [Method] We conduct a quantitative empirical study where the object of study, a DSML in the domain of ethical, social and environmental accounting, is supported by a metamodel in UML and a textual grammar in Xtext. Ninety-two participants grouped in 25 teams have evolved the DSML based on a set of new requirements, updating the metamodel and the grammar. We assess the completeness of each evolution activity and identify incompletions per artefact. We have also enquired the participants about their perceptions of the evolution process. [Results] The completeness of the metamodel evolution activity is about 1.25 times higher than it is for the grammar. The metamodelling primitives that are more likely to cause problems are relationships and enumerations. With respect to the Xtext grammars most incompletions are localised in rule calls, cross references and cardinalities. This is consistent with the participants' perceptions about the difficulty of each activity and primitive. [Contribution] Our findings are relevant for the design and testing of DSMLs, as well as for education on DSMLs.