Classifying legal interpretations using large language models

In the civil law tradition, legal arguments are used to justify the outcomes of judicial decision-making. These arguments are formed relying on a canon of interpretation techniques (e.g. textual or teleological interpretation). We study the identifiability of interpretation techniques as they are employed by the European Court of Human Rights (ECtHR) from a computational law perspective using a unique dataset. We show how Large Language Models (LLMs) can be utilized to classify legal interpretations, and we compare their performance. We evaluate proprietary and opensource models using methods such as few-shot and zero-shot chain-of-thought prompting combined with self-consistency. Our results imply that feature-extraction using LLMs leads to robust outcomes while allowing for greater resource- and timeefficiency compared to human annotation. Furthermore, our results imply that LLMs can play a larger role in the extraction of more complex features that are of particular relevance from a legal perspective.