Self-Collaboration Code Generation via ChatGPT

被引:6
作者
Dong, Yihong [1 ,2 ]
Jiang, Xue [1 ,2 ]
Jin, Zhi [1 ,2 ]
Li, Ge [1 ,2 ]
机构
[1] Peking Univ, Key Lab High Confidence Software Technol, Minist Educ, Beijing, Peoples R China
[2] Peking Univ, Sch Comp Sci, Beijing, Peoples R China
基金
中国国家自然科学基金;
关键词
Code generation; large language models; multi-agent collaboration; software development;
D O I
10.1145/3672459
中图分类号
TP31 [计算机软件];
学科分类号
081202 ; 0835 ;
摘要
Although large language models (LLMs) have demonstrated remarkable code-generation ability, they still struggle with complex tasks. In real-world software development, humans usually tackle complex tasks through collaborative teamwork, a strategy that significantly controls development complexity and enhances software quality. Inspired by this, we present a self-collaboration framework for code generation employing LLMs, exemplified by ChatGPT. Specifically, through role instructions, (1) Multiple LLM agents act as distinct "experts," each responsible for a specific subtask within a complex task; (2) Specify the way to collaborate and interact, so that different roles form a virtual team to facilitate each other's work, ultimately the virtual team addresses code generation tasks collaboratively without the need for human intervention. To effectively organize and manage this virtual team, we incorporate software-development methodology into the framework. Thus, we assemble an elementary team consisting of three LLM roles (i.e., analyst, coder, and tester) responsible for software development's analysis, coding, and testing stages. We conduct comprehensive experiments on various code-generation benchmarks. Experimental results indicate that self-collaboration code generation relatively improves 29.9-47.1% Pass@1 compared to the base LLM agent. Moreover, we showcase that selfcollaboration could potentially enable LLMs to efficiently handle complex repository-level tasks that are not readily solved by the single LLM agent.
引用
收藏
页数:38
相关论文
共 65 条
  • [51] Shin T, 2020, PROCEEDINGS OF THE 2020 CONFERENCE ON EMPIRICAL METHODS IN NATURAL LANGUAGE PROCESSING (EMNLP), P4222
  • [52] THE SOCIETY OF MIND - MINSKY,M
    SMOLIAR, SW
    [J]. ARTIFICIAL INTELLIGENCE, 1991, 48 (03) : 349 - 370
  • [53] Tai CY, 2023, Arxiv, DOI [arXiv:2305.14215, DOI 10.48550/ARXIV.2305.14215]
  • [54] THUDM, 2023, ChatGLM
  • [55] Touvron H, 2023, Arxiv, DOI arXiv:2307.09288
  • [56] Wei JS, 2022, ADV NEUR IN
  • [57] Wei JS, 2022, Arxiv, DOI arXiv:2201.11903
  • [58] Wu CF, 2023, Arxiv, DOI [arXiv:2303.04671, 10.48550/arXiv.2303.04671]
  • [59] Xia C. S., 2023, arXiv
  • [60] Xia CS, 2024, Arxiv, DOI [arXiv:2210.14179, DOI 10.48550/ARXIV.2210.14179]