Increasing Accessibility of Language Models with Multi-stage Information Extraction

The capabilities of Language Models (LMs) have continued to increase in recent years, as have their computational requirements. Widely available APIs have also become available. These APIs present new challenges for ease of gradient based fine-tuning by users, resulting in the use models which may be larger than necessary and more expensive, therefore reducing accessibility. In this paper, we present a new methodology for increasing performance of single-shot LMs by chaining multiple smaller LMs. Additionally, as the derived representation is in plain-text it is readily human interpretable. We show that optimizing the context which leads to this derived representation results in improved performance and reduced cost.