dongdongshe / k-scheduler Goto Github PK

Hi, I carefully read the K-Scheduler paper, especially the Evaluation section. I have a few questions about the Fuzzbench.

1. What are the advantages of using the Fuzzbench for edge coverage experiments? Because many works (such as ecofuzz, TortoiseFuzz, etc.) used real-world applications in their papers instead. Is there any difference between using programs in Fuzzbench and downloading the corresponding programs directly from the official website?
2. I would like to ask whether K-Scheduler runs the Fuzzbench's programs in the docker or compiles the Fuzzbench's programs outside the docker.
3. If K-Scheduler fuzzes the target programs in the docker, how to configure the related scripts to extract the ICFG of the program for K-Scheduler? If the program in Fuzzbench is deployed outside the docker, what process and standard should be followed to compile the program in Fuzzbench? (For example, what program version should be chosen, and why does harfbuzz need to compile the wrapper additionally?)

Log

The inter-CFG generation on project gpac failed with the following log:

+ python3 /home/qiuhongjun/AlphaFuzz-Experiment/fuzzers/kscheduler/afl_integration/build_example/gen_graph.py ./MP4Box_fix.ll cfg_out_MP4Box
Traceback (most recent call last):
  File "/home/qiuhongjun/AlphaFuzz-Experiment/fuzzers/kscheduler/afl_integration/build_example/gen_graph.py", line 206, in <module>
    inline_table = inline_counter_table(sys.argv[1])
  File "/home/qiuhongjun/AlphaFuzz-Experiment/fuzzers/kscheduler/afl_integration/build_example/gen_graph.py", line 28, in inline_counter_table
    line = subprocess.check_output('grep "llvm.compiler.used" ' + filename, shell=True, encoding='utf-8')[:-1]
  File "/usr/lib/python3.8/subprocess.py", line 415, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
  File "/usr/lib/python3.8/subprocess.py", line 516, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command 'grep "llvm.compiler.used" ./MP4Box_fix.ll' returned non-zero exit status 1.

POC

MP4Box_fix.ll.zip
All related files(Google Drive)

Hi, I use k-scheduler to fuzz xmllint2.6.0. I build xmllint.elf successfully but fail to execute it. Specifically, I can use afl-clang-fast to build xmllint2.6.0 and use afl to fuzz it successfully. So I think maybe there’s something wrong with K-scheduler. Can you help me to figure out the problem? I use K-scheduler to fuzz other programs(bsdtar, nasm, etc) successfully.

libxml2-2.6.0.zip

When I execute the last command in step 4.Build harfbuzz following Google FuzzBench settings $CXX $CXXFLAGS -std=c++11 -I ~/harfbuzz_BUILD/src/ ~/harfbuzz_BUILD/test/fuzzing/hb-fuzzer.o ~/harfbuzz_BUILD/src/.libs/libharfbuzz-fuzzing.a afl_llvm_rt_driver.a -o harfbuzz_afl_asan about Run K-Scheduler-based afl on an example program harfbuzz，it tells me clang-11: error: no such file or directory: '/home/jq/harfbuzz_BUILD/src/.libs/libharfbuzz-fuzzing.a'.
Can you help me?

请问为什么不能运行？

Hi, Im using K-scheduler to fuzz several programs(such as sqlite, tiff2pdf, tiff2bw), I want to test the performance of the K-scheduler by repeating my experiment for 10 times on each target. But i dont know how to set graph analysis module when I running multiple K-scheduler at the same time.
According to your issue, I opened 10 shells to run gen_dyn_weight.py, but how do I make each python script correspond to each k-scheduler that is running?

Hi.I successfully ran k-scheduler on the server and did some preliminary experiments. However I have a question about the results.

1. Results: The coverage performance of k-scheduler on some programs is not very good.
   As shown in the figure below, I counted the edge coverage results obtained by running AFL and k-scheduler on nm-new and readelf for 24 hours. I repeated the experiment for 20 times. However, I found that the results of k-scheduler and afl on nm-new program are comparable, but on readelf, the results obtained by k-scheduler are quite different from afl.

2. Question: I want to know what is the reason for this phenomenon? First of all, I know that AFL itself is advanced enough, so it is impossible to require a fuzzer to perform better than afl in all programs or scenarios. Besides, nm-new and readelf are compiled by the same version of binutils. I don't quite understand the difference between the edge coverage results of k-schdeduler on readelf and the edge coverage results of k-scheduler on other programs. Because I also tested some other programs, the edge coverage of k-scheduler on some programs is higher than AFL, but the result of k-scheduler on readelf makes me unable to understand.
3. Guess: I suspected at first that this result with a large deviation was caused by my misoperation. But when I use k-scheduler to fuzz 16 target programs, the configuration and command options of k-scheduler are the same. My running command and configuration are as follows:

command: 

aft-fuzz command:

./afl-fuzz_kscheduler -i AFL/testcases/others/elf/ -o res/24/readelf/kscheduler_20 -t 2000 -m none -d  testcases/kscheduler/binutils-2.38/binutils/readelf -a @@
./afl-fuzz_kscheduler -i AFL/testcases/others/elf/ -o res/24/nm-new/kscheduler_20 -t 2000 -m none -d  testcases/kscheduler/binutils-2.38/binutils/nm-new -C @@

python command: 

python3 gen_dyn_weight.py

The folder of my k-scheduler is shown in the figure below. Each folder has an afl-fuzz-kscheduler and gen_dyn_weight.py, in addition to the tested target program and the image file of the target program.

My server has a total of 100 logical cores, and I allocated 40 cores to the fuzzing task. When I run k-scheduler, I first start the gen_dyn_weight script, and then start the fuzz process:

So I would like to ask did you have encountered similar problems during the experiment? How did you solve it? What is the cause of this problem? Is it my configuration problem? Or does the system environment, such as the number of fd already opened in the system, affect the results of k-scheduler?

Hi, there

The gen_graph.py failed to generate a graph for some binaries in libtiff.

• command
  python3 /home/kscheduler/afl_integration/build_example/gen_graph.py ./tiff2pdf_fix.ll cfg_out_tiff2pdf

• Issue1

Traceback (most recent call last):
  File "/home/kscheduler/afl_integration/build_example/gen_graph.py", line 206, in <module>
    inline_table = inline_counter_table(sys.argv[1])
  File "/home/kscheduler/afl_integration/build_example/gen_graph.py", line 30, in inline_counter_table
    data[0] = data[0].split(' [i8*')[1]
IndexError: list index out of range

I added a if branch "if data != []: to fix this problem, and then face the following issue:

• Issue2

Traceback (most recent call last):
  File "/home/kscheduler/afl_integration/build_example/gen_graph.py", line 251, in <module>
    max_score = max(k.scores())
ValueError: max() arg is an empty sequence

• POC
  tiff2pdf.zip

I followed the instructions on https://github.com/Dongdongshe/K-Scheduler/blob/main/afl_integration/build_example/README.md to build harfbuzz with K-scheduler, and running the gen_graph.py script resulted in an error:

Traceback (most recent call last):
  File "/home/zenong/fuzz/K-Scheduler/afl_integration/build_example/gen_graph.py", line 250, in <module>
    max_score = max(k.scores())
ValueError: max() arg is an empty sequence

I checked the IR file and made sure it was compiled with LLVM coverage sanitizer ( __sanitizer_cov_trace_pc_guard exist in the IR).
I used Python3.10 with the following networkit version.

zenong@3ac4dd221bf1:~/fuzz$ python3 -m pip install networkit
Requirement already satisfied: networkit in /home/zenong/.local/lib/python3.10/site-packages (10.0)
Requirement already satisfied: numpy in /home/zenong/.local/lib/python3.10/site-packages (from networkit) (1.24.2)
Requirement already satisfied: scipy in /home/zenong/.local/lib/python3.10/site-packages (from networkit) (1.10.1)

Hi, I'm using afl_integration/build_example to compile harfbuzz with the following error when it comes to make statement, can you give me some suggestions to solve the problem?

make: Entering directory '/home/canicula/K-Scheduler/afl_integration/build_example/BUILD/src'
GEN libharfbuzz-fuzzing.la
libtool: error: cannot build libtool library 'libharfbuzz-fuzzing.la' from non-libtool objects on this host: /home/canicula/K-Scheduler/afl_integration/build_example/afl-llvm-rt.o
make: *** [Makefile:1542: libharfbuzz-fuzzing.la] Error 1
make: Leaving directory '/home/canicula/K-Scheduler/afl_integration/build_example/BUILD/src'

Hello, I have tested and read some K-Scheduler implementation source codes, and I have some questions to ask

1. During the experiment, it was found that the execution speed of using afl-fuzz_kscheduler is generally slower than that of using afl-fuzz. In the edge_log file, it is found that the graph centrality is calculated every 6 minutes on average. Where could the discrepancy be when there is no calculation
2. I don't know much about the instrumentation method of -fsanitize-coverage=trace-pc-guard. The default should be edge coverage.Cur_coverage should store edge coverage information, but these edge information are directly used in gen_dyn_weight.py to delete covered nodes. I would like to ask about the specific implementation here.

Hi there,

I have read your paper. K-scheduler is indeed an interesting work. I'm now trying to run the afl integration of K-scheduler following the instructions given in your document. According to the scripts attached bellow, it seems I should start one process to run katz computation and another process to run fuzzing. Do I have to run multiple katz computation instances if I want to run multiple fuzzing instances at once? If so, how to make different fuzzing instances to identify their own katz computation process?

cd [path to K-Schduler repo]/K-Scheduler/qsym_integration/build_example/
# clean fuzzer corpus and other meta data generated by fuzzer
rm -rf afl_out_* cur_coverage dyn_katz_cent
# reset signal file for graph computation module
echo 0 > signal
# run libfuzzer_kscheduler
./afl-fuzz_kscheduler -i seeds/ -o afl_out_cent -d -m none ./size @@

你好，我用k-scheduler来测试binutils，但是在调用gen_graph.py生成全局CFG时发现readelf、objdump的IR中并没有llvm.compiler.used。想问一下gen_graph.py只能为C++程序生成inter procedure CFG吗？

Hi there,

I have compiled and generate graphs for binutils2.37 and faust based on the README(non_wrapper based program).md.

For some binaries objdump and faust, segmentation fault appears when runningafl-fuzz_kscheduler. (But it goes well with binaries nm-new and readelf).

GDB

➜  base_debug git:(main) ✗ gdb /home/xxx/fuzzers/kscheduler/afl_integration/build_example/afl-fuzz_kscheduler
GNU gdb (Ubuntu 9.2-0ubuntu1~20.04) 9.2
Copyright (C) 2020 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later [http://gnu.org/licenses/gpl.html](http://gnu.org/licenses/gpl.html)
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
[http://www.gnu.org/software/gdb/bugs/](http://www.gnu.org/software/gdb/bugs/).
Find the GDB manual and other documentation resources online at:
[http://www.gnu.org/software/gdb/documentation/](http://www.gnu.org/software/gdb/documentation/).

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /home/xxx/fuzzers/kscheduler/afl_integration/build_example/afl-fuzz_kscheduler...
(gdb) r -i fuzz_in/ -o fuzz_out -d -m none -- ./objdump -D  @@
Starting program: /home/xxx/fuzzers/kscheduler/afl_integration/build_example/afl-fuzz_kscheduler -i fuzz_in/ -o fuzz_out -d -m none -- ./objdump -D  @@
afl-fuzz 2.52b by [[email protected]](mailto:[email protected])
[+] You have 128 CPU cores and 55 runnable tasks (utilization: 43%).
[+] Try parallel jobs - see docs/parallel_fuzzing.txt.
[*] Checking CPU core loadout...
[+] Found a free CPU core, binding to #53.
[*] Checking core_pattern...
[*] Setting up output directories...
[*] Scanning 'fuzz_in/'...
[+] No auto-generated dictionary tokens to reuse.
[*] Creating hard links for all input files...
[*] Validating target binary...
[*] Attempting dry run with 'id:000000,orig:small_exec.elf'...
[*] Spinning up the fork server...
[Detaching after fork from child process 3741177]
[+] All right - fork server is up.

Program received signal SIGSEGV, Segmentation fault.
classify_counts (mem=0x7ffff7c74fe8) at afl-fuzz.c:1195
1195    afl-fuzz.c: No such file or directory.
(gdb) backtrace
#0  classify_counts (mem=0x7ffff7c74fe8) at afl-fuzz.c:1195
#1  run_target (argv=<optimized out>, timeout=<optimized out>) at afl-fuzz.c:2711
#2  0x00005555555652c7 in calibrate_case_dry_run (handicap=0, from_queue=1 '\001',
use_mem=<optimized out>, q=0x5555564002e8, argv=<optimized out>) at afl-fuzz.c:3069
#3  perform_dry_run (argv=0x7fffffffdf60) at afl-fuzz.c:3234
#4  0x0000555555558b5d in main (argc=12, argv=0x7fffffffdf18) at afl-fuzz.c:8577