A Survey of Text Generation and Evaluation Based on Intrinsic Quality Constraints

被引：0

作者：

Lan Y.-Q. ^{[1
]}

Rao Y. ^{[1
]}

Li G.-C. ^{[2
]}

Sun L. ^{[1
]}

Xia B.-C. ^{[1
]}

Xin T.-T. ^{[1
]}

机构：

[1] Laboratory of Social Intelligence & Complex Data Processing, School of Software Engineering, Xi’an Jiaotong University, Shaanxi, Xi’an

[2] Beijing China Changfeng Electromechanical Technology Research and Design Institute, Beijing

来源：

Tien Tzu Hsueh Pao/Acta Electronica Sinica | 2024年 / 52卷 / 02期

基金：

中国国家自然科学基金;

关键词：

language model; natural language processing; text evaluation; text generation; text quality;

D O I：

10.12263/DZXB.20230826

中图分类号：

学科分类号：

摘要：

Recently, the outstanding text generation language models represented by ChatGPT, which can adapt to complex scenes and meet various application demands of human beings, has become the focuses of both the academic and industrial circles. However, the advantage of large language models (LLM) such as ChatGPT that are highly faithful to user intent implies some factual errors, and it is also necessary to rely on prompt content to control the detailed generation quality and domain adaptability, so it is still of great significance to study text generation with intrinsic quality constraints as the core. Based on the comparative study of key content generation models and technologies in recent years, this paper defined the basic form of text generation with intrinsic quality constraints, and six quality features based on“credibility, expressiveness and elegance”. In view of these 6 quality features, we provided analysis and comparison of generator mod⁃ el design and related algorithms. Besides, various automatic and human evaluation methods for different intrinsic quality features are summarized. Finally, this paper looks forward to the future research directions of intrinsic quality constraint technology. © 2024 Chinese Institute of Electronics. All rights reserved.

引用

页码：633 / 659

页数：26

共 41 条

[1] OUYANG, WU J, JIANG X, Et al., Training language models to follow instructions with human feedback [EB/OL]
[2] SCHULMAN J, ZOPH B, KIM C, Et al., ChatGPT: Opti⁃ mizing language models for dialogue
[3] BROWN T B, MANN B, RYDER N, Et al., Language mod⁃ els are few-shot learners, Proceedings of the 34th Inter⁃ national Conference on Neural Information Processing Sys⁃ tems, pp. 1877-1901, (2020)
[4] LEWIS M, LIU Y H, GOYAL N, Et al., BART: denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension, Proceed⁃ ings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 7871-7880, (2020)
[5] TURING A M., Computing machinery and intelligence [M], Parsing the Turing Test, pp. 23-65, (2009)
[6] ZHANG M., An inquiry into Yan fu’s translation theory of faithfulness, expressiveness, and elegance: The beginning of China’s modern translation theory, Trans-Humanities Journal, 6, 3, pp. 179-196, (2013)
[7] RAFFEL C, SHAZEER N, ROBERTS A, Et al., Exploring the limits of transfer learning with a unified text-to-text transformer
[8] BUBECK S, CHANDRASEKARAN V, ELDAN R, Et al., Sparks of Artificial General Intelligence: Early experi⁃ ments with GPT-4
[9] VASWANI A, SHAZEER N, PARMAR N, Et al., Atten⁃ tion is all you need
[10] DEVLIN J, CHANG M-W, LEE K, Et al., Bert: Pre-train⁃ ing of deep bidirectional transformers for language under⁃ standing, Proceedings of the 2019 Conference of the North American Chapter of the Association for Computa⁃ tional Linguistics: Human Language Technologies, Vol⁃ ume 1 (Long and Short Papers), pp. 4171-4186, (2019)

← 1 2 3 4 5 →