Contrastive learning with large language models for medical code prediction

Medical code prediction is an important method for unstructured electronic health records (EHRs) analysis. It is a disease classification method introduced by the Worldwide Health Organization (WHO) that can associate EMR with the corresponding medical code. The challenges of automatic medical code prediction mainly include the long clinical text and long-tailed label distribution. Prior studies have designed pre-trained language models (PLM) for medical code prediction. However, These methods either did not solve the problem of input text length limit, or they did not propose additional methods for better code representations to solve the problem of long-tailed label distribution. In this work, we propose a novel contrastive learning framework with a large language model (CL-LLM) for medical code prediction. CL-LLM exploits the large language model (LLM) in solving the long-tailed label distribution problem by designing a prompt method to generate synonymous code descriptions. In addition, CL-LLM uses contrastive learning to inject synonyms into code description encoder to enhance the model's few-shot label prediction. To solve the problem of long input text, we perform PLMClinicalBERT as a clinical text encoder and split pooling to segment long input text. We conducted experiments on the public MIMIC-III and MIMIC-IV datasets. The results on MIMIC III and MIMIC-IV datasets show that our model outperforms previous state-of-the-art methods for automatic ICD coding. We also conducted ablation experiments to prove the importance of each component in CL-LLM. To further verify the performance on rare labels, we test our methods on the MIMIC-III RARE50 dataset and achieve predominant results.