Towards Predicting Source Code Changes Based on Natural Language Processing Models: An Empirical Evaluation

In this paper, we investigate the prediction of software code changes using a natural language processing (NLP) model. NLP is one of the most rapidly developing fields in recent years, allowing various tasks related to natural language to be performed using large-scale models. In particular, BERT (bidirectional encoder representations from transformers) is a well-known model for encoding the input sentences of natural language into an appropriate vector space and is used for various classification tasks. In this paper, we use CuBERT (code understanding BERT), which was trained on programming languages as data in the pre-training stage, to perform tasks related to program code. Specifically, we run a regression problem where the output is the number of code changes.