Dr.PathFinder: hybrid fuzzing with deep reinforcement concolic execution toward deeper path-first search

被引：0
作者：
Seungho Jeon
Jongsub Moon
机构：
[1] Korea University,Division of Information Security, Graduate School of Information Security
来源：
Neural Computing and Applications | 2022年 / 34卷
关键词：
Fuzzing; Symbolic execution; Concolic execution; Reinforcement learning; Deep Q-network;
D O I：
暂无
中图分类号：
学科分类号：
摘要：
Fuzzing is an effective approach to discover bugs in programs, especially memory corruption bugs, using randomly generated test cases. However, without prior knowledge of the target program, the fuzzer can generate only a limited number of test cases because of sanity checks. To solve this problem, recent studies have proposed hybrid fuzzers that observe the context of a target program using symbolic execution; these fuzzers generate test cases to bypass the sanity check. While hybrid fuzzers explore “deep” bugs in the target program, they generate many ineffective test cases. In this paper, we propose a concolic execution algorithm that combines deep reinforcement learning with a hybrid fuzzing solution, Dr.PathFinder. When the reinforcement learning agent encounters a branch during concolic execution, it evaluates the state and determines the search path. In this process,“shallow” paths are pruned, and “deep” paths are searched first. This reduces unnecessary exploration, allowing the efficient memory usage and alleviating the state explosion problem. In experiments with the CB-multios dataset for deep bug cases, Dr.PathFinder discovered approximately five times more bugs than AFL and two times more than Driller-AFL. In addition to finding more bugs, Dr.PathFinder generated 19 times fewer test cases and used at least 2%\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$2\%$$\end{document} less memory than Driller-AFL. While it performed well in finding bugs located in deep paths, Dr.PathFinder had limitation to find bugs located at shallow paths, which we discussed.
引用
收藏
页码：10731 / 10750
页数：19
相关论文
共 92 条
  • [1] Böhme M(2016)Coverage-based greybox fuzzing as Markov chain Proc ACM Conf Comput Commun Secur 1 2-17
  • [2] Pham VT(2014)TaintDroid: an information-flow tracking system for realtime privacy monitoring on smartphones ACM Trans Comput Syst 1 2-undefined
  • [3] Roychoudhury A(2012)DTAM: dynamic taint analysis of multi-threaded programs for relevancy Proc ACM SIGSOFT Int Symp Found Softw Eng 33 1-undefined
  • [4] Enck W(2009)Taint-based directed whitebox fuzzing Proc Int Conf Softw Eng undefined undefined-undefined
  • [5] Gilbert P(2005)DART: directed automated random testing ACM SIGPLAN Not undefined undefined-undefined
  • [6] Han S(1997)Long short-term memory Neural Comput undefined undefined-undefined
  • [7] Tendulkar V(1992)Undecidability of static analysis ACM Lett Program Lang Syst undefined undefined-undefined
  • [8] Chun BG(2020)DeepFuzzer: accelerated deep Greybox fuzzing IEEE Trans Depend Secur Comput undefined undefined-undefined
  • [9] Cox LP(2015)Human-level control through deep reinforcement learning Nature undefined undefined-undefined
  • [10] Jung J(2016)Mastering the game of Go with deep neural networks and tree search Nature undefined undefined-undefined
12345678910