Evaluating Code Comment Generation With Summarized API Docs

被引:0
|
作者
Matmti, Bilel [1 ]
Fard, Fatemeh [1 ]
机构
[1] Univ British Columbia, Dept Comp Sci, Okanagan, BC, Canada
来源
2023 IEEE/ACM 2ND INTERNATIONAL WORKSHOP ON NATURAL LANGUAGE-BASED SOFTWARE ENGINEERING, NLBSE | 2023年
基金
加拿大自然科学与工程研究理事会;
关键词
API Docs; text summarization; comment generation; external knowledge source;
D O I
10.1109/NLBSE59153.2023.00019
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Code comment generation is the task of generating a high-level natural language description for a given code snippet. API2Com is a comment generation model designed to leverage the Application Programming Interface Documentations (API Docs) as an external knowledge resource. Shahbazi et al. [1] showed that API Docs might help increase the model's performance. However, the model's performance in generating pertinent comments deteriorates due to the lengthy documentation used in the input as the number of APIs used in a method increases. In this paper, we propose to evaluate how summarizing the API Docs using an extractive text summarization technique, TextRank, will impact the overall performance of the API2Com. The results of our experiments using the same Java dataset confirm the inverse correlation between the number of APIs and the model's performance. As the number of APIs increases, the performance metrics tend to deteriorate for both configurations of the model, with or without API Docs summarization using TextRank. Experiments also show the impact of the number of APIs on TextRank algorithm capacity to improve the model performance. For example, with 8 APIs, TextRank summarization improved the model BLEU score by 18% on average, but the performance tends to decrease as the number of APIs increases. This demonstrates an open area of research to determine the winning combination in terms of the model configuration and the length of documentation used.
引用
收藏
页码:60 / 63
页数:4
相关论文
共 21 条
  • [21] MESIA: Understanding and Leveraging Supplementary Nature of Method-level Comments for Automatic Comment Generation
    Pan, Xinglu
    Liu, Chenxiao
    Zou, Yanzhen
    Xie, Tao
    Xie, Bing
    PROCEEDINGS 2024 32ND IEEE/ACM INTERNATIONAL CONFERENCE ON PROGRAM COMPREHENSION, ICPC 2024, 2024, : 74 - 86