Competitions for Benchmarking Task and Functionality Scoring Complete Performance Assessment

被引：40

作者：

Amigoni, Francesco ^{[1
]}

Bastianelli, Emanuele ^{[2
]}

Berghofer, Jakob ^{[3
]}

Bonarini, Andrea ^{[1
]}

Fontana, Giulio ^{[1
]}

Hochgeschwender, Nico ^{[4
]}

Iocchi, Luca ^{[2
]}

Kraetzschmar, Gerhard K. ^{[4
]}

Lima, Pedro ^{[5
]}

Matteucci, Matteo ^{[1
]}

Miraldo, Pedro ^{[5
]}

Nardi, Daniele ^{[2
]}

Schiaffonati, Viola ^{[1
]}

机构：

[1] Politecn Milan, Milan, Italy

[2] Univ Roma La Sapienza, Rome, Italy

[3] KUKA Labs GmbH, Augsburg, Germany

[4] Bonn Rhein Sieg Univ, St Augustin, Germany

[5] Univ Lisbon, Inst Super Tecn, P-1699 Lisbon, Portugal

来源：

IEEE ROBOTICS & AUTOMATION MAGAZINE | 2015年 / 22卷 / 03期

关键词：

ROBOT COMPETITIONS; CHALLENGES;

D O I：

10.1109/MRA.2015.2448871

中图分类号：

TP [自动化技术、计算机技术];

学科分类号：

0812 ;

摘要：

Scientific experiments and robotic competitions share some common traits that can put the debate about developing better experimental methodologies and replicability of results in robotics research on more solid ground. In this context, the Robot Competitions Kick Innovation in Cognitive Systems and Robotics (RoCKIn) project aims to develop competitions that come close to scientific experiments, providing an objective performance evaluation of robot systems under controlled and replicable conditions. In this article, by further articulating replicability into reproducibility and repeatability and by considering some results from the 2014 first RoCKIn competition, we show that the RoCKIn approach offers tools that enable the replicability of experimental results. © 2015 IEEE.

引用

页码：53 / 61

页数：9

共 18 条

[1]

Ahmad A., 2014, ROCKIN DELIVERABLE D

[2] Recasting Robotics Challenges as Experiments [J].

Anderson, Monica ;

Jenkins, Odest Chadwicke ;

Osentoski, Sarah .

IEEE ROBOTICS & AUTOMATION MAGAZINE, 2011, 18 (02) :10-11

[3]

Bonasso P, 1997, AI MAG, V18, P11

[4]

Bonsignorio F., 2007, SPECIAL INTEREST GRO

[5] Research relevance of mobile robot competitions [J].

Bräunl, T .

IEEE ROBOTICS & AUTOMATION MAGAZINE, 1999, 6 (04) :32-37

[6]

Collberg C., 2014, Measuring reproducibility in computer systems research

[7]

Drummond Chris, 2009, 4 WORKSH EV METH MAC

[8]

Dwiputra R., 2014, P ISR ROB, P328

[9]

Hacking I., 1983, REPRESENTING INTERVE

[10]

Hernandez-Orallo Jose, 2014, AI EVALUATION PRESEN

← 1 2 →