Is a comparison of results meaningful from the inexact replications of computational experiments?

被引:57
作者
Crepinsek, Matej [1 ]
Liu, Shih-Hsi [2 ]
Mernik, Luka [3 ]
Mernik, Marjan [1 ]
机构
[1] Univ Maribor, SLO-2000 Maribor, Slovenia
[2] Calif State Univ Fresno, Fresno, CA 93740 USA
[3] CALTECH, Pasadena, CA 91125 USA
关键词
Teaching-learning-based optimization; Algorithm comparison; Replication of experiments; LEARNING-BASED OPTIMIZATION; EVOLUTIONARY ALGORITHMS; PARAMETER CONTROL; EXPLOITATION; EXPLORATION;
D O I
10.1007/s00500-014-1493-4
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
The main objective of this paper is to correct the unreasonable and inaccurate criticism to our previous experiments using Teaching-Learning-Based Optimization algorithm and to quantify the amount of error that may arise due to incorrect counting of fitness evaluations. It is shown that inexact experiment replication should be avoided in comparisons between meta-heuristic algorithms whenever possible. Otherwise, an inexact replication and margin of error should be explicitly reported.
引用
收藏
页码:223 / 235
页数:13
相关论文
共 34 条
[1]  
[Anonymous], 2013, Evolutionary Optimization Algorithms
[2]  
[Anonymous], SIGACT NEWS
[3]  
[Anonymous], 2000, 6 INT C PAR PROBL SO
[4]   An Overview of Evolutionary Algorithms for Parameter Optimization [J].
Baeck, Thomas ;
Schwefel, Hans-Paul .
EVOLUTIONARY COMPUTATION, 1993, 1 (01) :1-23
[5]  
Barr R. S., 1995, Journal of Heuristics, V1, P9, DOI 10.1007/BF02430363
[6]   Metaheuristics in combinatorial optimization: Overview and conceptual comparison [J].
Blum, C ;
Roli, A .
ACM COMPUTING SURVEYS, 2003, 35 (03) :268-308
[7]  
Crepinsek Matej, 2011, International Innovative Computing and Applications, V3, P11, DOI 10.1504/IJICA.2011.037947
[8]   Exploration and Exploitation in Evolutionary Algorithms: A Survey [J].
Crepinsek, Matej ;
Liu, Shih-Hsi ;
Mernik, Marjan .
ACM COMPUTING SURVEYS, 2013, 45 (03)
[9]   A note on teaching-learning-based optimization algorithm [J].
Crepinsek, Matej ;
Liu, Shih-Hsi ;
Mernik, Luka .
INFORMATION SCIENCES, 2012, 212 :79-93
[10]   A practical tutorial on the use of nonparametric statistical tests as a methodology for comparing evolutionary and swarm intelligence algorithms [J].
Derrac, Joaquin ;
Garcia, Salvador ;
Molina, Daniel ;
Herrera, Francisco .
SWARM AND EVOLUTIONARY COMPUTATION, 2011, 1 (01) :3-18