An Empirical Study of the Intuitive Understanding of a Formal Pattern Language

[Context and motivation] Formal pattern languages with a restricted English grammar, such as the pattern language of Konrad and Cheng, give us the possibility to combine human intuition and the rigour of a machine. [Question/problem] The question arises to what extent the intuitive understanding of such a pattern language is in agreement with its formal semantics. [Principal ideas/results] We present an empirical study to address this question. The existence of a formal semantics allows us to use the machine as an objective judge to decide if the intuitive understanding is correct. The study confirms empirically the practical usefulness of HANFORPL in that the intuitive understanding matches the formal semantics in most practically relevant cases. The study reveals that a number of phrases of interest represent critical edge cases where even a prior exposure to formal logic is not a guarantee for the correct intuitive understanding. [Contribution] We show how the alignment of formal and intuitive semantics can be investigated, and that this alignment can not simply be assumed. Nonetheless, results regarding the understandability of HANFORPL are favourable with high understandability in commonly used patterns. The results of the study will be the basis of improvements in HANFORPL.