WiRe57: A Fine-Grained Benchmark for Open Information Extraction

被引：0

作者：

Lechelle, William ^{[1
]}

Gotti, Fabrizio ^{[1
]}

Langlais, Philippe ^{[1
]}

机构：

[1] Univ Montreal, RALI, Montreal, PQ, Canada

来源：

13TH LINGUISTIC ANNOTATION WORKSHOP (LAW XIII) | 2019年

关键词：

D O I：

暂无

中图分类号：

TP18 [人工智能理论];

学科分类号：

081104 ; 0812 ; 0835 ; 1405 ;

摘要：

We build a reference for the task of Open Information Extraction, on five documents. We tentatively resolve a number of issues that arise, including coreference and granularity, and we take steps toward addressing inference, a significant problem. We seek to better pinpoint the requirements for the task. We produce our annotation guidelines specifying what is correct to extract and what is not. In turn, we use this reference to score existing Open IE systems. We address the nontrivial problem of evaluating the extractions produced by systems against the reference tuples, and share our evaluation script. Among seven compared extractors, we find the MinIE system to perform best.

引用

页码：6 / 15

页数：10

共 24 条

[1]

Angeli G., 2013, P CONLL, P133

[2]

[Anonymous], 2015, ACL

[3]

[Anonymous], 2015, P 2015 C EMP METH NA, DOI 10.18653/v1/D15-1076

[4]

[Anonymous], 2013, P INT WWW C WWW, DOI DOI 10.1145/2488388.2488420

[5] Open Information Extraction via Contextual Sentence Decomposition [J].

Bast, Hannah ;

Haussmann, Elmar .

2013 IEEE SEVENTH INTERNATIONAL CONFERENCE ON SEMANTIC COMPUTING (ICSC 2013), 2013, :154-159

[6]

Bhutani N., 2016, P 2016 C EMP METH NA, P55, DOI DOI 10.18653/V1/D16-1006

[7]

Christensen J., 2011, K CAP, P113

[8]

de Sa Mesquita Filipe, 2013, P 2013 C EMP METH NA, P447

[9]

Fader A., 2011, Identifying Relations for Open Information Extraction

[10] Open Question Answering Over Curated and Extracted Knowledge Bases [J].

Fader, Anthony ;

Zettlemoyer, Luke ;

Etzioni, Oren .

PROCEEDINGS OF THE 20TH ACM SIGKDD INTERNATIONAL CONFERENCE ON KNOWLEDGE DISCOVERY AND DATA MINING (KDD'14), 2014, :1156-1165

← 1 2 3 →