"I'm Sorry, but I Can't Assist": Bias in Generative AI

被引：1

作者：

Smith, Julie M. ^{[1
]}

机构：

[1] Inst Adv Comp Educ, Peoria, IL 61629 USA

来源：

PROCEEDINGS OF THE CONFERENCE FOR RESEARCH ON EQUITABLE AND SUSTAINED PARTICIPATION IN ENGINEERING, COMPUTING, AND TECHNOLOGY, RESPECT 2024 | 2024年

基金：

美国国家科学基金会;

关键词：

equity; student advising; artificial intelligence; large language models; generative AI; racism;

D O I：

10.1145/3653666.3656065

中图分类号：

G40 [教育学];

学科分类号：

040101 ; 120403 ;

摘要：

Research Questions: (1) Is there a pattern of racial bias in student advising recommendations made by generative AI? (2) What safeguards can promote equity when using generative AI in high-stakes decision-making? Methodology: Using lists of names associated with various ethnic/racial groups, we asked ChatGPT and Claude AI for recommendations for colleges and majors for each student. Results: ChatGPT was more likely to recommend STEM majors to some student groups. ChatGPT did not show systematic bias in various metrics of school quality, but Claude AI did. There were also overall differences in the colleges recommended by Claude AI and ChatGPT. Implications: We provide cautions and recommendations for using generative AI in high-stakes tasks.

引用

页码：75 / 80

页数：6

共 37 条

[1] AI-Supported Academic Advising: Exploring ChatGPT's Current State and Future Potential toward Student Empowerment [J].

Akiba, Daisuke ;

Fraboni, Michelle C. .

EDUCATION SCIENCES, 2023, 13 (09)

[2]

[Anonymous], 2022, Introducing ChatGPT

[3]

[Anonymous], 2020, What is the Average SAT Score?

[4]

anthropic, Introducing Claude

[5]

Assayed Suha Khalil, 2023, A Generative AI Chatbot in High School Advising: A Qualitative Analysis of Domain-Specific Chatbot and ChatGPT

[6] Diagnostic Accuracy of a Large Language Model in Pediatric Case Studies [J].

Barile, Joseph ;

Margolis, Alex ;

Cason, Grace ;

Kim, Rachel ;

Kalash, Saia ;

Tchaconas, Alexis ;

Milanaik, Ruth .

JAMA PEDIATRICS, 2024, 178 (03) :313-315

[7]

Bhaskara Vinay, Dear Superintendents, Get Ready For the AI Guidance Revolution

[8]

Capgemini, 2023, Technical Report

[9]

Clark Rick, 2023, Counselors, Can We ChatGPT?

[10]

Department of Homeland Security, DHS STEM Designated Degree Program List

← 1 2 3 4 →