Stop Word Lists in Free Open-source Software Packages

被引：0
|
作者：
Nothman, Joel [1 ]
Qin, Hanmin [2 ]
Yurchak, Roman [3 ]
机构：
[1] Univ Sydney, Sydney Informat Hub, Sydney, NSW, Australia
[2] Peking Univ, Beijing, Peoples R China
[3] Symerio, Palaiseau, France
来源：
NLP OPEN SOURCE SOFTWARE (NLP-OSS) | 2018年
关键词：
D O I：
暂无
中图分类号：
TP18 [人工智能理论];
学科分类号：
081104 ; 0812 ; 0835 ; 1405 ;
摘要：
Open-source software (OSS) packages for natural language processing often include stop word lists. Users may apply them without awareness of their surprising omissions (e.g. hasn't but not hadn't) and inclusions (e.g. computer), or their incompatibility with particular tokenizers. Motivated by issues raised about the Scikit-learn stop list, we investigate variation among and consistency within 52 popular English-language stop lists, and propose strategies for mitigating these issues.
引用
收藏
页码：7 / 12
页数：6
相关论文
共 50 条
  • [1] Free and Open-Source Software in Higher Education
    Oreski, Predrag
    Kermek, Dragutin
    Kirinic, Valentina
    CROATIAN JOURNAL OF EDUCATION-HRVATSKI CASOPIS ZA ODGOJ I OBRAZOVANJE, 2024, 26 (03): : 823 - 879
  • [2] Opportunity Costs in Free Open-Source Software
    Karus, Siim
    OPEN SOURCE SYSTEMS, OSS 2019, 2019, 556 : 139 - 150
  • [3] The Free Software Alternative: Freeware, Open-source software, and Libraries
    Corbly, James E.
    INFORMATION TECHNOLOGY AND LIBRARIES, 2014, 33 (03) : 65 - 75
  • [4] Osirix: Free and open-source software for medical imagery
    Jalbert, F.
    Paoli, J. R.
    REVUE DE STOMATOLOGIE ET DE CHIRURGIE MAXILLO-FACIALE, 2008, 109 (01): : 53 - 55
  • [5] An overview about Geographic free and open-source software
    Dabove, Paolo
    GEAM-GEOINGEGNERIA AMBIENTALE E MINERARIA-GEAM-GEOENGINEERING ENVIRONMENT AND MINING, 2019, (157): : 11 - 14
  • [6] STRATEGIC FORKING IN THE DEVELOPMENT OF FREE/OPEN-SOURCE SOFTWARE
    Konat, Grzegorz
    GOSPODARKA NARODOWA, 2008, (10): : 23 - 43
  • [7] Defining software evolvability from a free/open-source software perspective
    Deprez, Jean-Christophe
    Monfils, Frederic Fleurial
    Ciolkowski, Marcus
    Soto, Martin
    THIRD INTERNATIONAL IEEE WORKSHOP ON SOFTWARE EVOLVABILITY, PROCEEDINGS, 2007, : 29 - +
  • [8] Financing A Free-for-All: Crowdfunding Open-Source Software
    Bort, James
    Meoli, Azzurra
    Sullivan, David W.
    SMALL BUSINESS ECONOMICS, 2024, 63 (04) : 1449 - 1469
  • [9] Specvis: Free and open-source software for visual field examination
    Dzwiniel, Piotr
    Gola, Mateusz
    Wojcik-Grycluk, Anna
    Waleszczyk, Wioletta J.
    PLOS ONE, 2017, 12 (10):
  • [10] Open-source software - Introduction
    Sabbah, D
    Frye, D
    IBM SYSTEMS JOURNAL, 2005, 44 (02)
12345