dongdongshe / k-scheduler Goto Github PK
View Code? Open in Web Editor NEWA universal seed scheduler for fuzzers (LibFuzzer and AFL havoc mode) and concolic execution engine (qsym).
License: MIT License
A universal seed scheduler for fuzzers (LibFuzzer and AFL havoc mode) and concolic execution engine (qsym).
License: MIT License
export PATH=[path to K-Scheduler repo]/K-Scheduler/libfuzzer_integration/llvm_11.0.1/build/bin
Hi, I use k-scheduler to fuzz xmllint2.6.0. I build xmllint.elf successfully but fail to execute it. Specifically, I can use afl-clang-fast to build xmllint2.6.0 and use afl to fuzz it successfully. So I think maybe there’s something wrong with K-scheduler. Can you help me to figure out the problem? I use K-scheduler to fuzz other programs(bsdtar, nasm, etc) successfully.
libxml2-2.6.0.zip
你好,我用k-scheduler来测试binutils,但是在调用gen_graph.py生成全局CFG时发现readelf、objdump的IR中并没有llvm.compiler.used。想问一下gen_graph.py只能为C++程序生成inter procedure CFG吗?
Hello, I have tested and read some K-Scheduler implementation source codes, and I have some questions to ask
Hi, I carefully read the K-Scheduler paper, especially the Evaluation section. I have a few questions about the Fuzzbench.
When I execute the last command in step 4.Build harfbuzz following Google FuzzBench settings $CXX $CXXFLAGS -std=c++11 -I ~/harfbuzz_BUILD/src/ ~/harfbuzz_BUILD/test/fuzzing/hb-fuzzer.o ~/harfbuzz_BUILD/src/.libs/libharfbuzz-fuzzing.a afl_llvm_rt_driver.a -o harfbuzz_afl_asan
about Run K-Scheduler-based afl on an example program harfbuzz,it tells me clang-11: error: no such file or directory: '/home/jq/harfbuzz_BUILD/src/.libs/libharfbuzz-fuzzing.a'
.
Can you help me?
Hi there,
I have read your paper. K-scheduler is indeed an interesting work. I'm now trying to run the afl integration of K-scheduler following the instructions given in your document. According to the scripts attached bellow, it seems I should start one process to run katz computation and another process to run fuzzing. Do I have to run multiple katz computation instances if I want to run multiple fuzzing instances at once? If so, how to make different fuzzing instances to identify their own katz computation process?
cd [path to K-Schduler repo]/K-Scheduler/qsym_integration/build_example/
# clean fuzzer corpus and other meta data generated by fuzzer
rm -rf afl_out_* cur_coverage dyn_katz_cent
# reset signal file for graph computation module
echo 0 > signal
# run libfuzzer_kscheduler
./afl-fuzz_kscheduler -i seeds/ -o afl_out_cent -d -m none ./size @@
Hi there,
I have compiled and generate graphs for binutils2.37 and faust based on the README(non_wrapper based program).md.
For some binaries objdump and faust, segmentation fault appears when runningafl-fuzz_kscheduler
. (But it goes well with binaries nm-new and readelf).
➜ base_debug git:(main) ✗ gdb /home/xxx/fuzzers/kscheduler/afl_integration/build_example/afl-fuzz_kscheduler
GNU gdb (Ubuntu 9.2-0ubuntu1~20.04) 9.2
Copyright (C) 2020 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later [http://gnu.org/licenses/gpl.html](http://gnu.org/licenses/gpl.html)
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
[http://www.gnu.org/software/gdb/bugs/](http://www.gnu.org/software/gdb/bugs/).
Find the GDB manual and other documentation resources online at:
[http://www.gnu.org/software/gdb/documentation/](http://www.gnu.org/software/gdb/documentation/).
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /home/xxx/fuzzers/kscheduler/afl_integration/build_example/afl-fuzz_kscheduler...
(gdb) r -i fuzz_in/ -o fuzz_out -d -m none -- ./objdump -D @@
Starting program: /home/xxx/fuzzers/kscheduler/afl_integration/build_example/afl-fuzz_kscheduler -i fuzz_in/ -o fuzz_out -d -m none -- ./objdump -D @@
afl-fuzz 2.52b by [[email protected]](mailto:[email protected])
[+] You have 128 CPU cores and 55 runnable tasks (utilization: 43%).
[+] Try parallel jobs - see docs/parallel_fuzzing.txt.
[*] Checking CPU core loadout...
[+] Found a free CPU core, binding to #53.
[*] Checking core_pattern...
[*] Setting up output directories...
[*] Scanning 'fuzz_in/'...
[+] No auto-generated dictionary tokens to reuse.
[*] Creating hard links for all input files...
[*] Validating target binary...
[*] Attempting dry run with 'id:000000,orig:small_exec.elf'...
[*] Spinning up the fork server...
[Detaching after fork from child process 3741177]
[+] All right - fork server is up.
Program received signal SIGSEGV, Segmentation fault.
classify_counts (mem=0x7ffff7c74fe8) at afl-fuzz.c:1195
1195 afl-fuzz.c: No such file or directory.
(gdb) backtrace
#0 classify_counts (mem=0x7ffff7c74fe8) at afl-fuzz.c:1195
#1 run_target (argv=<optimized out>, timeout=<optimized out>) at afl-fuzz.c:2711
#2 0x00005555555652c7 in calibrate_case_dry_run (handicap=0, from_queue=1 '\001',
use_mem=<optimized out>, q=0x5555564002e8, argv=<optimized out>) at afl-fuzz.c:3069
#3 perform_dry_run (argv=0x7fffffffdf60) at afl-fuzz.c:3234
#4 0x0000555555558b5d in main (argc=12, argv=0x7fffffffdf18) at afl-fuzz.c:8577
The inter-CFG generation on project gpac failed with the following log:
+ python3 /home/qiuhongjun/AlphaFuzz-Experiment/fuzzers/kscheduler/afl_integration/build_example/gen_graph.py ./MP4Box_fix.ll cfg_out_MP4Box
Traceback (most recent call last):
File "/home/qiuhongjun/AlphaFuzz-Experiment/fuzzers/kscheduler/afl_integration/build_example/gen_graph.py", line 206, in <module>
inline_table = inline_counter_table(sys.argv[1])
File "/home/qiuhongjun/AlphaFuzz-Experiment/fuzzers/kscheduler/afl_integration/build_example/gen_graph.py", line 28, in inline_counter_table
line = subprocess.check_output('grep "llvm.compiler.used" ' + filename, shell=True, encoding='utf-8')[:-1]
File "/usr/lib/python3.8/subprocess.py", line 415, in check_output
return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
File "/usr/lib/python3.8/subprocess.py", line 516, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command 'grep "llvm.compiler.used" ./MP4Box_fix.ll' returned non-zero exit status 1.
I followed the instructions on https://github.com/Dongdongshe/K-Scheduler/blob/main/afl_integration/build_example/README.md to build harfbuzz with K-scheduler, and running the gen_graph.py script resulted in an error:
Traceback (most recent call last):
File "/home/zenong/fuzz/K-Scheduler/afl_integration/build_example/gen_graph.py", line 250, in <module>
max_score = max(k.scores())
ValueError: max() arg is an empty sequence
I checked the IR file and made sure it was compiled with LLVM coverage sanitizer ( __sanitizer_cov_trace_pc_guard
exist in the IR).
I used Python3.10 with the following networkit version.
zenong@3ac4dd221bf1:~/fuzz$ python3 -m pip install networkit
Requirement already satisfied: networkit in /home/zenong/.local/lib/python3.10/site-packages (10.0)
Requirement already satisfied: numpy in /home/zenong/.local/lib/python3.10/site-packages (from networkit) (1.24.2)
Requirement already satisfied: scipy in /home/zenong/.local/lib/python3.10/site-packages (from networkit) (1.10.1)
Hi, there
The gen_graph.py
failed to generate a graph for some binaries in libtiff.
command
python3 /home/kscheduler/afl_integration/build_example/gen_graph.py ./tiff2pdf_fix.ll cfg_out_tiff2pdf
Issue1
Traceback (most recent call last):
File "/home/kscheduler/afl_integration/build_example/gen_graph.py", line 206, in <module>
inline_table = inline_counter_table(sys.argv[1])
File "/home/kscheduler/afl_integration/build_example/gen_graph.py", line 30, in inline_counter_table
data[0] = data[0].split(' [i8*')[1]
IndexError: list index out of range
I added a if branch "if data != []:
to fix this problem, and then face the following issue:
Traceback (most recent call last):
File "/home/kscheduler/afl_integration/build_example/gen_graph.py", line 251, in <module>
max_score = max(k.scores())
ValueError: max() arg is an empty sequence
Hi, Im using K-scheduler to fuzz several programs(such as sqlite, tiff2pdf, tiff2bw), I want to test the performance of the K-scheduler by repeating my experiment for 10 times on each target. But i dont know how to set graph analysis module when I running multiple K-scheduler at the same time.
According to your issue, I opened 10 shells to run gen_dyn_weight.py, but how do I make each python script correspond to each k-scheduler that is running?
Hi, I'm using afl_integration/build_example to compile harfbuzz with the following error when it comes to make statement, can you give me some suggestions to solve the problem?
make: Entering directory '/home/canicula/K-Scheduler/afl_integration/build_example/BUILD/src'
GEN libharfbuzz-fuzzing.la
libtool: error: cannot build libtool library 'libharfbuzz-fuzzing.la' from non-libtool objects on this host: /home/canicula/K-Scheduler/afl_integration/build_example/afl-llvm-rt.o
make: *** [Makefile:1542: libharfbuzz-fuzzing.la] Error 1
make: Leaving directory '/home/canicula/K-Scheduler/afl_integration/build_example/BUILD/src'
Hi.I successfully ran k-scheduler on the server and did some preliminary experiments. However I have a question about the results.
command:
aft-fuzz command:
./afl-fuzz_kscheduler -i AFL/testcases/others/elf/ -o res/24/readelf/kscheduler_20 -t 2000 -m none -d testcases/kscheduler/binutils-2.38/binutils/readelf -a @@
./afl-fuzz_kscheduler -i AFL/testcases/others/elf/ -o res/24/nm-new/kscheduler_20 -t 2000 -m none -d testcases/kscheduler/binutils-2.38/binutils/nm-new -C @@
python command:
python3 gen_dyn_weight.py
The folder of my k-scheduler is shown in the figure below. Each folder has an afl-fuzz-kscheduler and gen_dyn_weight.py, in addition to the tested target program and the image file of the target program.
My server has a total of 100 logical cores, and I allocated 40 cores to the fuzzing task. When I run k-scheduler, I first start the gen_dyn_weight script, and then start the fuzz process:
So I would like to ask did you have encountered similar problems during the experiment? How did you solve it? What is the cause of this problem? Is it my configuration problem? Or does the system environment, such as the number of fd
already opened in the system, affect the results of k-scheduler?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.