Challenges of Testing Machine Learning Based Systems

被引:34
|
作者
Marijan, Dusica [1 ]
Gotlieb, Arnaud [1 ]
Ahuja, Mohit Kumar [1 ]
机构
[1] Simula Res Lab, Fornebu, Norway
来源
2019 IEEE INTERNATIONAL CONFERENCE ON ARTIFICIAL INTELLIGENCE TESTING (AITEST) | 2019年
关键词
software testing; machine learning; machine learning based testing; testing of machine learning based systems;
D O I
10.1109/AITest.2019.00010
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Machine learning is being increasingly deployed in many industrial and societal applications. Given its widespread use, it is of utmost importance to ensure the quality of software systems supported by machine learning. Although researchers have applied some concepts from traditional software testing to testing of machine learning based systems, the latter introduce a range of challenges not typical for traditional software systems, thus making traditional software testing techniques ineffective. In this paper, we discuss the challenges intrinsic to testing of machine learning based systems. We highlight the promising role of machine learning based testing to alleviate some of these challenges. We also discuss directions for future research in this domain. This paper focuses on the testing aspects of machine learning based systems from the quality assurance perspective, rather than model performance perspective.
引用
收藏
页码:101 / 102
页数:2
相关论文
共 50 条
  • [1] Testing machine learning based systems: a systematic mapping
    Riccio, Vincenzo
    Jahangirova, Gunel
    Stocco, Andrea
    Humbatova, Nargiz
    Weiss, Michael
    Tonella, Paolo
    EMPIRICAL SOFTWARE ENGINEERING, 2020, 25 (06) : 5193 - 5254
  • [2] Grammar Based Directed Testing of Machine Learning Systems
    Udeshi, Sakshi
    Chattopadhyay, Sudipta
    IEEE TRANSACTIONS ON SOFTWARE ENGINEERING, 2021, 47 (11) : 2487 - 2503
  • [3] Testing machine learning based systems: a systematic mapping
    Vincenzo Riccio
    Gunel Jahangirova
    Andrea Stocco
    Nargiz Humbatova
    Michael Weiss
    Paolo Tonella
    Empirical Software Engineering, 2020, 25 : 5193 - 5254
  • [4] A Machine Learning Based Methodology for Web Systems Codeless Testing with Selenium
    Nguyen, Phuc
    Maag, Stephane
    SOFTWARE TECHNOLOGIES (ICSOFT 2020), 2021, 1447 : 184 - 202
  • [5] Challenges for machine learning in cooperative information systems
    Singh, MP
    Huhns, MN
    DISTRIBUTED ARTIFICIAL INTELLIGENCE MEETS MACHINE LEARNING: LEARNING IN MULTI-AGENT ENVIRONMENTS, 1997, 1221 : 11 - 24
  • [6] Testing machine learning systems in real estate
    Wan, Wayne Xinwei
    Lindenthal, Thies
    REAL ESTATE ECONOMICS, 2023, 51 (03) : 754 - 778
  • [7] Search-based fairness testing for regression-based machine learning systems
    Perera, Anjana
    Aleti, Aldeida
    Tantithamthavorn, Chakkrit
    Jiarpakdee, Jirayus
    Turhan, Burak
    Kuhn, Lisa
    Walker, Katie
    EMPIRICAL SOFTWARE ENGINEERING, 2022, 27 (03)
  • [8] Search-based fairness testing for regression-based machine learning systems
    Anjana Perera
    Aldeida Aleti
    Chakkrit Tantithamthavorn
    Jirayus Jiarpakdee
    Burak Turhan
    Lisa Kuhn
    Katie Walker
    Empirical Software Engineering, 2022, 27
  • [9] Metamorphic Testing For Machine Learning: Applicability, Challenges, and Research Opportunities
    Rehman, Faqeer Ur
    Srinivasan, Madhusudan
    2023 IEEE INTERNATIONAL CONFERENCE ON ARTIFICIAL INTELLIGENCE TESTING, AITEST, 2023, : 34 - 39
  • [10] Data Management in Machine Learning: Challenges, Techniques, and Systems
    Kumar, Arun
    Boehm, Matthias
    Yang, Jun
    SIGMOD'17: PROCEEDINGS OF THE 2017 ACM INTERNATIONAL CONFERENCE ON MANAGEMENT OF DATA, 2017, : 1717 - 1722