Making It Possible for the Auditing of AI: A Systematic Review of AI Audits and AI Auditability

被引:0
作者
Li, Yueqi [1 ]
Goel, Sanjay [2 ]
机构
[1] Skidmore Coll, Management & Business Dept, 815 North Broadway, Saratoga Springs, NY 12866 USA
[2] Univ Albany State Univ New York, Sch Business, Dept Informat Secur & Digital Forens, 1400 Washington Ave, Albany, NY 12222 USA
关键词
Artificial intelligence (AI); AI audits; Auditability; Accountability; Transparency; Explainability; ARTIFICIAL-INTELLIGENCE; INFORMATION-SYSTEMS; QUALITY; CHALLENGES; ETHICS; WORLD;
D O I
10.1007/s10796-024-10508-8
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
Artificial intelligence (AI) technologies have become the key driver of innovation in society. However, numerous vulnerabilities of AI systems can lead to negative consequences for society, such as biases encoded in the training data and algorithms and lack of transparency. This calls for AI systems to be audited to ensure that the impact on society is understood and mitigated. To enable AI audits, auditability measures need to be implemented. This study provides a systematic review of academic work and regulatory work on AI audits and AI auditability. Results reveal the current understanding of the AI audit scope, audit challenges, and auditability measures. We identify and categorize AI auditability measures for each audit area and specific process to be audited and the party responsible for each process to be audited. Our findings will guide existing efforts to make AI systems auditable across the lifecycle of AI systems.
引用
收藏
页数:31
相关论文
共 157 条
[11]  
[Anonymous], 2022, Auditing algorithms: the existing landscape, role of regulators and future outlook
[12]   FactSheets: Increasing trust in AI services through supplier's declarations of conformity [J].
Arnold, M. ;
Bellamy, R. K. E. ;
Hind, M. ;
Houde, S. ;
Mehta, S. ;
Mojsilovic, A. ;
Nair, R. ;
Ramamurthy, K. Natesan ;
Olteanu, A. ;
Piorkowski, D. ;
Reimer, D. ;
Richards, J. ;
Tsay, J. ;
Varshney, K. R. .
IBM JOURNAL OF RESEARCH AND DEVELOPMENT, 2019, 63 (4-5)
[13]   Assuring the Machine Learning Lifecycle: Desiderata, Methods, and Challenges [J].
Ashmore, Rob ;
Calinescu, Radu ;
Paterson, Colin .
ACM COMPUTING SURVEYS, 2021, 54 (05)
[14]   Integrating a Blockchain-Based Governance Framework for Responsible AI [J].
Asif, Rameez ;
Hassan, Syed Raheel ;
Parr, Gerard .
FUTURE INTERNET, 2023, 15 (03)
[15]  
Ayling J., 2022, AI Ethics, V2, P405, DOI [DOI 10.1007/S43681-021-00084-X, 10.1007/s43681-021-00084-x]
[16]  
Balahur A., 2022, In JRC Conference and Workshop, DOI [10.2760/365479, DOI 10.2760/365479]
[17]   AI Decision Making with Dignity? Contrasting Workers' Justice Perceptions of Human and AI Decision Making in a Human Resource Management Context [J].
Bankins, Sarah ;
Formosa, Paul ;
Griep, Yannick ;
Richards, Deborah .
INFORMATION SYSTEMS FRONTIERS, 2022, 24 (03) :857-875
[18]   Designing Disaggregated Evaluations of AI Systems: Choices, Considerations, and Tradeoffs [J].
Barocas, Solon ;
Guo, Anhong ;
Kamar, Ece ;
Krones, Jacquelyn ;
Morris, Meredith Ringel ;
Vaughan, Jennifer Wortman ;
Wadsworth, W. Duncan ;
Wallach, Hanna .
AIES '21: PROCEEDINGS OF THE 2021 AAAI/ACM CONFERENCE ON AI, ETHICS, AND SOCIETY, 2021, :368-378
[19]  
Bell Andrew, 2022, FAccT '22: 2022 ACM Conference on Fairness, Accountability, and Transparency, P248, DOI 10.1145/3531146.3533090
[20]   Bridging the Gap Between Ethics and Practice: Guidelines for Reliable, Safe, and Trustworthy Human-centered AI Systems [J].
Ben Shneiderman .
ACM TRANSACTIONS ON INTERACTIVE INTELLIGENT SYSTEMS, 2020, 10 (04)