Enhancing fuzzing efficacy : an in-depth exploration and development of fuzzing strategies

Abstract

While fuzzing has demonstrated its value in everyday use, there is a lack of foundational research in both the academic and industrial sectors regarding various critical mechanisms of fuzzing. This deficiency leaves researchers without the necessary guidance when advancing new research related to fuzzing. Therefore, we determined to conduct comprehensive studies about the fundamental components of fuzzing to obtain more insights for further research. We first evaluate Havoc (first implemented in AFL [178]) since it is widely adopted in many existing fuzzers as a fundamental fuzzing strategy. We further propose HavocMAB with a new guidance algorithm according to our findings to schedule mutators automatically from the exploration history. Next, we study the effectiveness of existing gradient-based fuzzers and propose new fuzzers based on new guidance algorithms named PreFuzz. Through our previous research, we also have found that the fundamental guidance mechanism, coverage guidance, can be less effective when fuzzing deep program states of the target programs. Moreover, random fuzzing strategies can explore the target program efficiently, e.g., only adopting Havoc can already outperform tons of other approaches significantly. On the contrary, the random fuzzing strategies could be less effective in some scenarios. An application with complicated constraints can easily terminate at the early stage and thus fuzzers with random fuzzing strategies cannot explore it efficiently. To this end, we first propose the concept of phantom program, which is built to mitigate the over-compliance of program dependencies to improve the efficiency of coverage guidance. Accordingly, we build a coverage-guided fuzzer namely MirageFuzz which performs dual fuzzing for the original program and the phantom program simultaneously and adopts the taint-based mutation mechanism to generate new mutants by combining the resulting seeds from dual fuzzing via taint analysis. The evaluation results show that MirageFuzz outperforms the baseline fuzzers from 13.42% to 77.96% in terms of edge coverage averagely in our benchmark. Secondly, we focus on testing the large-scale, constraint-laden Java Virtual Machine (JVM) to explore how to conduct random exploration without violating constraints. Simultaneously, we also investigate whether there are alternative approaches to coverage guidance when dealing with excessively large target programs. We first propose a coverage-guided fuzzing framework, namely JITfuzz, to automatically detect JIT bugs. JITfuzz adopts a set of optimization-activating mutators to trigger the usage of typical JIT optimizations. Meanwhile, JITfuzz also adopts mutators to enrich the control flows of target programs. To date, JITfuzz detects 36 unknown JVM bugs and 27 of them have been confirmed by the developers. Next, we propose SJFuzz, which employs a discrepancy-guided seed scheduler to retain discrepancy-inducing class files and class files that generate discrepancy-inducing mutants for fuzzing guidance. We have reported 46 previously unknown potential issues discovered by SJFuzz to the JVM developers where 20 were confirmed as bugs and 16 were fixed. Our empirical studies have delivered critical insights for future fuzzing research and our proposed techniques inspired by our studies have successfully facilitated fuzzing efficacy. In the future, we plan to improve testing efficacy in more challenging fields, e.g., CPU/GPU testing.