Evaluating skills in hierarchical reinforcement learning

被引：0
作者：
Marzieh Davoodabadi Farahani
Nasser Mozayani
机构：
[1] Iran University of Science and Technology,Computer Engineering Department
来源：
International Journal of Machine Learning and Cybernetics | 2020年 / 11卷
关键词：
Hierarchical reinforcement learning; Temporal abstraction; Option; Skill; Option evaluation;
D O I：
暂无
中图分类号：
学科分类号：
摘要：
Despite the benefits mentioned in previous works of automatically acquiring skills for using them in hierarchical reinforcement learning algorithms such as solving the curse of dimensionality, improving exploration, and speeding up value propagation, they have not paid much attention to evaluating the effect of each skill on these factors. In this paper, we show that depending on the given task, a skill may be useful for learning it or not. In addition, the focus of the related work of automatically acquiring skills is on detecting subgoals, i.e., the skill termination condition, but there is not a precise method for extracting the initiation set of skills. In this paper, we propose not only two methods for evaluating skills but also two other methods for pruning the initiation set of them. Experimental results show significant improvements in learning different test domains after evaluating and pruning skills.
引用
收藏
页码：2407 / 2420
页数：13
相关论文
共 23 条
  • [1] Lin LJ(1992)Self-improving reactive agents based on reinforcement learning, planning and teaching Mach Learn 8 293-321
  • [2] Sutton RS(1998)Reinforcement learning: an introduction IEEE Trans Neural Netw 9 1054-1054
  • [3] Barto AG(1999)Between MDPs and semi-MDPs: a framework for temporal abstraction in reinforcement learning Artif Intell 112 181-211
  • [4] Sutton RS(2017)Graph based skill acquisition and transfer learning for continuous reinforcement learning domains Pattern Recognit Lett 87 104-116
  • [5] Precup D(2017)Integrating skills and simulation to solve complex navigation tasks in Infinite Mario IEEE Trans Games 10 101-106
  • [6] Singh S(1996)Reinforcement learning: a survey J Artif Intell Res 4 237-285
  • [7] Shoeleh F(2019)Automatic construction and evaluation of macro-actions in reinforcement learning Appl Soft Comput 82 105574-141
  • [8] Asadpour M(2004)Finding and evaluating community structure in networks Phys Rev E 69 026113-96
  • [9] Dann M(2018)Proposing a new method for acquiring skills in reinforcement learning with the help of graph clustering Iran J Electr Comput Eng 2 131-354
  • [10] Zambetta F(2013)Learning graph-based representations for continuous reinforcement learning domains Mach Learn Knowl Discov Databases 8188 81-undefined
123