A Survey of Metrics to Enhance Training Dependability in Large Language Models

被引：0

作者：

Fang, Wenyi ^{[1
]}

Zhang, Hao ^{[1
]}

Gong, Ziyu ^{[1
]}

Zeng, Longbin ^{[1
]}

Lu, Xuhui ^{[1
,2
]}

Liu, Biao ^{[1
]}

Wu, Xiaoyu ^{[1
]}

Zheng, Yang ^{[1
]}

Hu, Zheng ^{[1
]}

Zhang, Xun ^{[1
]}

机构：

[1] Huawei Technol Co Ltd, Shenzhen, Peoples R China

[2] Beihang Univ, Sch Automat Sci & Elect Engn, Beijing, Peoples R China

来源：

2023 IEEE 34TH INTERNATIONAL SYMPOSIUM ON SOFTWARE RELIABILITY ENGINEERING WORKSHOPS, ISSREW | 2023年

关键词：

Large Language Model; Dependability; Monitoring Metric;

D O I：

10.1109/ISSREW60843.2023.00071

中图分类号：

TP18 [人工智能理论];

学科分类号：

081104 ; 0812 ; 0835 ; 1405 ;

摘要：

The rapidly advancing field of artificial intelligence requires meticulous attention to the training and monitoring of large language models (LLMs). This paper offers a systematic analysis of existing metrics and introduces new ones, focusing on their theoretical underpinnings and practical implementations. We present empirical results and insights into the performance of selected metrics, elucidating the complex interplay of variables in the training process. Our comprehensive approach provides significant insights into LLM training, and promises to improve the dependability and efficiency of future models.

引用

页码：180 / 185

页数：6

共 50 条

[21] A comprehensive survey on GNN-based anomaly detection: taxonomy, methods, and the role of large language models
Yuan, Ziqi
Sun, Qingyun
Zhou, Haoyi
Shao, Minglai
Fu, Xingcheng
INTERNATIONAL JOURNAL OF MACHINE LEARNING AND CYBERNETICS, 2025,
[22] Human-AI Synergy in Survey Development: Implications from Large Language Models in Business and Research
Fan, Ke ping
Chung, Ng ka
ACM TRANSACTIONS ON MANAGEMENT INFORMATION SYSTEMS, 2025, 16 (01) : 24 - 39
[23] Large language models for automated Q&A involving legal documents: a survey on algorithms, frameworks and applications
Yang, Xiaoxian
Wang, Zhifeng
Wang, Qi
Wei, Ke
Zhang, Kaiqi
Shi, Jiangang
INTERNATIONAL JOURNAL OF WEB INFORMATION SYSTEMS, 2024, 20 (04) : 413 - 435
[24] Large Language Models as Evaluators for Recommendation Explanations
Zhang, Xiaoyu
Li, Yishan
Wang, Jiayin
Sun, Bowen
Ma, Weizhi
Sun, Peijie
Zhang, Min
PROCEEDINGS OF THE EIGHTEENTH ACM CONFERENCE ON RECOMMENDER SYSTEMS, RECSYS 2024, 2024, : 33 - 42
[25] Prompting Large Language Models With the Socratic Method
Chang, Edward Y.
2023 IEEE 13TH ANNUAL COMPUTING AND COMMUNICATION WORKSHOP AND CONFERENCE, CCWC, 2023, : 351 - 360
[26] Improving Recommender Systems with Large Language Models
Lubos, Sebastian
ADJUNCT PROCEEDINGS OF THE 32ND ACM CONFERENCE ON USER MODELING, ADAPTATION AND PERSONALIZATION, UMAP 2024, 2024, : 40 - 44
[27] The use of large language models for program repair
Zubair, Fida
Al-Hitmi, Maryam
Catal, Cagatay
COMPUTER STANDARDS & INTERFACES, 2025, 93
[28] Game Generation via Large Language Models
Hu, Chengpeng
Zhao, Yunlong
Liu, Jialin
2024 IEEE CONFERENCE ON GAMES, COG 2024, 2024,
[29] Large Language Models: An Emerging Technology in Accounting
Vasarhelyi, Miklos A.
Moffitt, Kevin C.
Stewart, Trevor
Sunderland, Dan
JOURNAL OF EMERGING TECHNOLOGIES IN ACCOUNTING, 2023, 20 (02) : 1 - 10
[30] Enhancing Persona Consistency with Large Language Models
Shi, Haozhe
Niu, Kun
2024 5TH INTERNATIONAL CONFERENCE ON COMPUTING, NETWORKS AND INTERNET OF THINGS, CNIOT 2024, 2024, : 210 - 215

← 1 2 3 4 5 →