Hello all,

I could not get the tool to build, even though I closely followed the build instructions and used the same tool versions, compiler versions and OS as it was described in the documentation.

I got the following error:

Step 21/24 : RUN cd Superion/llvm_mode && LLVM_CONFIG=llvm-config-3.8 CXXFLAGS="-DLLVM38" make ---> Running in 6cd7db6eb15d [*] Checking for working 'llvm-config'... [*] Checking for working '/root/Superion/afl-gcc'... [*] Checking for '../afl-showmap'... [+] All set and ready to build. /root/Superion/afl-gcc -O3 -funroll-loops -Wall -D_FORTIFY_SOURCE=2 -g -Wno-pointer-sign -DAFL_PATH=\"/usr/local/lib/afl\" -DBIN_PATH=\"/usr/local/bin\" -DVERSION=\"2.52b\" afl-clang-fast.c -o ../afl-clang-fast ln -sf afl-clang-fast ../afl-clang-fast++ /root/Superion/afl-g++ llvm-config-3.8 --cxxflags-fno-rtti -fpic -DLLVM38 -Wall -D_FORTIFY_SOURCE=2 -g -Wno-pointer-sign -DVERSION=\"2.52b\" -Wno-variadic-macros -shared afl-llvm-pass.so.cc -o ../afl-llvm-pass.sollvm-config-3.8 --ldflags cc1plus: warning: command line option '-Wno-pointer-sign' is valid for C/ObjC but not for C++ afl-llvm-pass.so.cc: In member function 'virtual bool {anonymous}::AFLCoverage::runOnModule(llvm::Module&)': afl-llvm-pass.so.cc:180:28: warning: variable 'instLine' set but not used [-Wunused-but-set-variable] unsigned int instLine = cDILoc->getLine(); ^ /root/Superion/afl-gcc -O3 -funroll-loops -Wall -D_FORTIFY_SOURCE=2 -g -Wno-pointer-sign -DAFL_PATH=\"/usr/local/lib/afl\" -DBIN_PATH=\"/usr/local/bin\" -DVERSION=\"2.52b\" -fPIC -c afl-llvm-rt.o.c -o ../afl-llvm-rt.o afl-llvm-rt.o.c:249:1: warning: constructor priorities from 0 to 100 are reserved for the implementation __attribute__((constructor(CONST_PRIO))) void __afl_auto_init(void) { ^ /tmp/.afl-56-1563905864.s: Assembler messages: /tmp/.afl-56-1563905864.s:5041: Error: symbol__afl_area_ptr' is already defined
/tmp/.afl-56-1563905864.s:5042: Error: symbol __afl_prev_loc' is already defined Makefile:86: recipe for target '../afl-llvm-rt.o' failed make: *** [../afl-llvm-rt.o] Error 1 The command '/bin/sh -c cd Superion/llvm_mode && LLVM_CONFIG=llvm-config-3.8 CXXFLAGS="-DLLVM38" make' returned a non-zero code: 2

Everything is written in a Dockerfile which can be seen below. In order to build the docker I used the following command: docker build . --network=host

Prepare OS and dependencies

FROM ubuntu:16.04
WORKDIR /root/
COPY antlr-4.7.2-complete.jar ./
RUN apt-get update && apt-get install -y curl git cmake xz-utils
RUN curl http://releases.llvm.org/3.8.0/clang+llvm-3.8.0-x86_64-linux-gnu-ubuntu-16.04.tar.xz | tar xJf -
RUN apt-get update && apt-get install -y --no-install-recommends build-essential python3
RUN apt-get update && apt-get install -y --no-install-recommends apt-utils
RUN apt-get update && apt-get install -y --no-install-recommends wget software-properties-common
RUN apt-get update && apt-get install -y --no-install-recommends git
RUN apt-get install -y make cmake g++ pkg-config uuid-dev
RUN apt-add-repository "deb http://us.archive.ubuntu.com/ubuntu/ xenial universe" && apt-add-repository "deb http://us.archive.ubuntu.com/ubuntu/ xenial main restricted"
RUN apt-get update && apt-get install -y clang-3.8 libomp-dev libllvm3.8

Cloning Superion

RUN git clone https://github.com/zhunki/Superion

Build ANTLR runtime

RUN cd Superion/tree_mutation/ && cmake . && make -j2

Build tree mutator for JS and PHP

RUN cd Superion/tree_mutation/php_parser && for f in *.cpp; do g++ -I ../runtime/src/ -c $f -fPIC -std=c++11; done && g++ -shared -std=c++11 *.o ../dist/libantlr4-runtime.a -o libTreeMutation.so
RUN cd Superion/tree_mutation/js_parser && for f in *.cpp; do g++ -I ../runtime/src/ -c $f -fPIC -std=c++11; done && g++ -shared -std=c++11 *.o ../dist/libantlr4-runtime.a -o libTreeMutation.so

Build AFL

RUN cd Superion && make
ENV AFL_HARDEN=1
ENV CC=/root/Superion/afl-gcc
ENV CXX=/root/Superion/afl-g++
RUN cd Superion/llvm_mode && LLVM_CONFIG=llvm-config-3.8 CXXFLAGS="-DLLVM38" make

Cloning and building JerryScript

RUN git clone https://github.com/jerryscript-project/jerryscript.git
RUN cd jerryscript/tools && python build.py --clean --debug --compile-flag=-fsanitize=address --compile-flag=-m32 --compile-flag=-fno-omit-frame-pointer --compile-flag=-fno-common --jerry-libc=off --static-link=off --lto=off --error-message=on --system-allocator=on

Fuzzing JerryScript

RUN cd Superion && ./afl-fuzz -M f1 -m 1G -t 100+ -i ~/tree_mutation/js_parser/ -o ~/tree_mutation/js_parser/ ~/jerryscript/build/bin/jerry @@

I tried to use the js_parser. When I used the xml_parser everything works fine but when I use the js_parser it gives me the following error:

[-] PROGRAM ABORT : Symbol 'afl_custom_init' not found.
Location : load_custom_mutator(), src/afl-fuzz-mutators.c:198
I looked into the TreeMutation.cpp from the js_parser and it indeed does not have afl_custom_init while the TreeMutation.cpp in the xml_parser does have afl_custom_init.

This happens on startup of afl when the custom mutator library is being loaded.

Hi @zhunki

First of all, great work on this. I'd been working on and off on a syntax-aware fuzzer using this AST mutation approach myself. So, I'm glad you put in the foot-work to do this already.

Just a minor NIT here. I'd been receiving numerous Invalid Syntax errors while running afl-fuzz on an instrumented JavaScriptCore build, until I realized that the Makefile I'd built afl-fuzz with included a hard-coded reference to the PHP parser shared library. I suggest either (or both) of the following:

1. Making this parameter configurable within the Makefile with a default value
2. Updating the README.md to make it more apparent

If you'd rather I open a PR for this, let me know.

Hi,

I got AFL stability under < 50%, anyone else is having the same issue?

thanks.

Hi,

I tried to add new parsers by running

java -jar antlr-4.7.2-complete.jar -o path -visitor -no-listener -Dlanguage=SQLite SQLite.g4

But there is an error:

error(7):  cannot find or open file: SQLite.g4

Could you please give any reference on adding the new parsers to Superion?

Hi all.

how can I add a new grammar for a javascript web api? Thanks!

export CC=/Superion/afl-clang-fast && \
export CXX=/Superion/afl-clang-fast++ && \
export AFL_HARDEN=1 &&
git clone https://github.com/espruino/Espruino.git && \
cd Espruino && \
CC=/Superion/afl-clang-fast CXX=afl-clang-fast++ make

Everything works except for some reason

./afl-fuzz -M f1 -m 1G -t 100+ -i /seeds -o /out /Espruino/espruino @@

says that the binary is not instrumented?
Also for another JS engine I am getting errors about unrecognized flags? How would I fix that?

when I run

g++ -shared -std=c++11 *.o ../dist/libantlr4-runtime.a -o libTreeMutation.so
g++: error: ../dist/libantlr4-runtime.a: No such file or directory

The previous commands runs successfully, but gives the following warnings:

for f in *.cpp; do g++ -I ../runtime/src/ -c $f -fPIC -std=c++11; done In file included from ../runtime/src/atn/PredictionContext.h:10:0, from ../runtime/src/LexerInterpreter.h:9, from ../runtime/src/antlr4-runtime.h:32, from ECMAScriptBaseVisitor.h:8, from ECMAScriptBaseVisitor.cpp:5: ../runtime/src/atn/ATNState.h:73:26: warning: type attributes ignored after type is already defined [-Wattributes] class ANTLR4CPP_PUBLIC ATN; ^~~ In file included from ../runtime/src/atn/PredictionContext.h:10:0, from ../runtime/src/LexerInterpreter.h:9, from ../runtime/src/antlr4-runtime.h:32, from ECMAScriptLexer.h:7, from ECMAScriptLexer.cpp:5: ../runtime/src/atn/ATNState.h:73:26: warning: type attributes ignored after type is already defined [-Wattributes] class ANTLR4CPP_PUBLIC ATN; ^~~ In file included from ../runtime/src/atn/PredictionContext.h:10:0, from ../runtime/src/LexerInterpreter.h:9, from ../runtime/src/antlr4-runtime.h:32, from ECMAScriptVisitor.h:7, from ECMAScriptParser.cpp:5: ../runtime/src/atn/ATNState.h:73:26: warning: type attributes ignored after type is already defined [-Wattributes] class ANTLR4CPP_PUBLIC ATN; ^~~ In file included from ../runtime/src/atn/PredictionContext.h:10:0, from ../runtime/src/LexerInterpreter.h:9, from ../runtime/src/antlr4-runtime.h:32, from ECMAScriptSecondVisitor.h:8, from ECMAScriptSecondVisitor.cpp:5: ../runtime/src/atn/ATNState.h:73:26: warning: type attributes ignored after type is already defined [-Wattributes] class ANTLR4CPP_PUBLIC ATN; ^~~ In file included from ../runtime/src/atn/PredictionContext.h:10:0, from ../runtime/src/LexerInterpreter.h:9, from ../runtime/src/antlr4-runtime.h:32, from ECMAScriptVisitor.h:7, from ECMAScriptVisitor.cpp:5: ../runtime/src/atn/ATNState.h:73:26: warning: type attributes ignored after type is already defined [-Wattributes] class ANTLR4CPP_PUBLIC ATN; ^~~ In file included from ../runtime/src/atn/PredictionContext.h:10:0, from ../runtime/src/LexerInterpreter.h:9, from ../runtime/src/antlr4-runtime.h:32, from TreeMutation.cpp:3: ../runtime/src/atn/ATNState.h:73:26: warning: type attributes ignored after type is already defined [-Wattributes] class ANTLR4CPP_PUBLIC ATN; ^~~

Hi,

Follow my previous issue #8 , I added a parser SQLite.g4. And I changed the Makefile to compile it as:

afl-fuzz: afl-fuzz.c $(COMM_HDR) | test_x86
        $(CC) $(CFLAGS) [email protected] -o $@ $(LDFLAGS) ./tree_mutation/sql_parser/libTreeMutation.so

And I make:

zhangys@xxx:~/utility/superion$ make
[*] Checking for the ability to compile x86 code...
[+] Everything seems to be working, ready to compile.
cc -O3 -funroll-loops -Wall -D_FORTIFY_SOURCE=2 -g -Wno-pointer-sign -DAFL_PATH=\"/usr/local/lib/afl\" -DDOC_PATH=\"/usr/local/share/doc/afl\" -DBIN_PATH=\"/usr/local/bin\" afl-gcc.c -o afl-gcc -ldl
set -e; for i in afl-g++ afl-clang afl-clang++; do ln -sf afl-gcc $i; done
cc -O3 -funroll-loops -Wall -D_FORTIFY_SOURCE=2 -g -Wno-pointer-sign -DAFL_PATH=\"/usr/local/lib/afl\" -DDOC_PATH=\"/usr/local/share/doc/afl\" -DBIN_PATH=\"/usr/local/bin\" afl-fuzz.c -o afl-fuzz -ldl ./tree_mutation/sql_parser/libTreeMutation.so
/tmp/cc96fGic.o: In function `fuzz_one':
/home/zhangys/utility/superion/afl-fuzz.c:6124: undefined reference to `parse'
/home/zhangys/utility/superion/afl-fuzz.c:6132: undefined reference to `fuzz'
collect2: error: ld returned 1 exit status
Makefile:73: recipe for target 'afl-fuzz' failed
make: *** [afl-fuzz] Error 1

It gives an error. if we look at the parse and fuzz functions, they are implemented in tree_mutation/xxx_parser/TreeMutation.cpp.

Let's see .../js_parser/TreeMutation.cpp

#include <iostream>
#include <cstring>
#include "antlr4-runtime.h"
#include "ECMAScriptLexer.h"
#include "ECMAScriptParser.h"
#include "ECMAScriptBaseVisitor.h"
#include "ECMAScriptSecondVisitor.h"
....

If look at the TreeMutation.cpp under the folder of different parsers, we could see almost the same code.

How doI write the new TreeMutation.cpp for the new grammar?

Hi ,
so after 3-4 days of work, I managed to get this working after I downgrade my OS to ubuntu 16 and then slowly built on that and then upgraded to ubuntu 18.

now that I'm fuzzing jsc, how can I know for sure Tree mutation strategy is working?

Only the classical one have visual indicators like bit/byte flipping etc.....Thanks!

I am experiencing low stability (<50%), solely false positives (none of the crashes reproduce), and weirdly it says a whole cycle was finished in a short time, all while fuzzing jsc (release build).

Any clue to why?

Also, did you run it against the release or debug version of jsc? With or without asan?

Hi all,

I got llvm-config version 9, so I adapted the command a bit, as seen bellow.

LLVM_CONFIG=llvm-config-9 CXXFLAGS="-DLLVM9" make [*] Checking for working 'llvm-config'... [*] Checking for working 'clang'... [*] Checking for '../afl-showmap'... [+] All set and ready to build. clang -v -O3 -funroll-loops -v -Wall -D_FORTIFY_SOURCE=2 -g -Wno-pointer-sign -DAFL_PATH=\"/usr/local/lib/afl\" -DBIN_PATH=\"/usr/local/bin\" -DVERSION=\"2.52b\" afl-clang-fast.c -o ../afl-clang-fast clang version 9.0.0-2 (tags/RELEASE_900/final) Target: x86_64-pc-linux-gnu Thread model: posix InstalledDir: /usr/bin Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/7 Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/7.5.0 Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/8 Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/9 Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/7 Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/7.5.0 Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/8 Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/9 Selected GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/9 Candidate multilib: .;@m64 Selected multilib: .;@m64 "/usr/lib/llvm-9/bin/clang" -cc1 -triple x86_64-pc-linux-gnu -emit-obj -disable-free -disable-llvm-verifier -discard-value-names -main-file-name afl-clang-fast.c -mrelocation-model static -mthread-model posix -fmath-errno -masm-verbose -mconstructor-aliases -munwind-tables -fuse-init-array -target-cpu x86-64 -dwarf-column-info -debug-info-kind=limited -dwarf-version=4 -debugger-tuning=gdb -momit-leaf-frame-pointer -v -v -resource-dir /usr/lib/llvm-9/lib/clang/9.0.0 -D _FORTIFY_SOURCE=2 -D "AFL_PATH=\"/usr/local/lib/afl\"" -D "BIN_PATH=\"/usr/local/bin\"" -D "VERSION=\"2.52b\"" -internal-isystem /usr/local/include -internal-isystem /usr/lib/llvm-9/lib/clang/9.0.0/include -internal-externc-isystem /usr/include/x86_64-linux-gnu -internal-externc-isystem /include -internal-externc-isystem /usr/include -O3 -Wall -Wno-pointer-sign -fdebug-compilation-dir /home/adrian/Superion/llvm_mode -ferror-limit 19 -fmessage-length 0 -funroll-loops -fobjc-runtime=gcc -fdiagnostics-show-option -fcolor-diagnostics -vectorize-loops -vectorize-slp -faddrsig -o /tmp/afl-clang-fast-b3bde1.o -x c afl-clang-fast.c clang -cc1 version 9.0.0 based upon LLVM 9.0.0 default target x86_64-pc-linux-gnu ignoring nonexistent directory "/include" #include "..." search starts here: #include <...> search starts here: /usr/local/include /usr/lib/llvm-9/lib/clang/9.0.0/include /usr/include/x86_64-linux-gnu /usr/include End of search list. "/usr/bin/ld" -z relro --hash-style=gnu --build-id --eh-frame-hdr -m elf_x86_64 -dynamic-linker /lib64/ld-linux-x86-64.so.2 -o ../afl-clang-fast /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/crt1.o /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/crti.o /usr/bin/../lib/gcc/x86_64-linux-gnu/9/crtbegin.o -L/usr/bin/../lib/gcc/x86_64-linux-gnu/9 -L/usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu -L/lib/x86_64-linux-gnu -L/lib/../lib64 -L/usr/lib/x86_64-linux-gnu -L/usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../.. -L/usr/lib/llvm-9/bin/../lib -L/lib -L/usr/lib /tmp/afl-clang-fast-b3bde1.o -lgcc --as-needed -lgcc_s --no-as-needed -lc -lc -lgcc --as-needed -lgcc_s --no-as-needed /usr/bin/../lib/gcc/x86_64-linux-gnu/9/crtend.o /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/crtn.o ln -sf afl-clang-fast ../afl-clang-fast++ clang++ llvm-config-9 --cxxflags-fno-rtti -fpic -DLLVM9 -v -Wall -D_FORTIFY_SOURCE=2 -g -Wno-pointer-sign -DVERSION=\"2.52b\" -Wno-variadic-macros -shared afl-llvm-pass.so.cc -o ../afl-llvm-pass.sollvm-config-9 --ldflags clang version 9.0.0-2 (tags/RELEASE_900/final) Target: x86_64-pc-linux-gnu Thread model: posix InstalledDir: /usr/bin Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/7 Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/7.5.0 Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/8 Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/9 Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/7 Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/7.5.0 Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/8 Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/9 Selected GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/9 Candidate multilib: .;@m64 Selected multilib: .;@m64 "/usr/lib/llvm-9/bin/clang" -cc1 -triple x86_64-pc-linux-gnu -emit-obj -mrelax-all -disable-free -disable-llvm-verifier -discard-value-names -main-file-name afl-llvm-pass.so.cc -mrelocation-model pic -pic-level 1 -mthread-model posix -mdisable-fp-elim -fmath-errno -masm-verbose -mconstructor-aliases -munwind-tables -fuse-init-array -target-cpu x86-64 -dwarf-column-info -debug-info-kind=limited -dwarf-version=4 -debugger-tuning=gdb -v -resource-dir /usr/lib/llvm-9/lib/clang/9.0.0 -I /usr/lib/llvm-9/include -D _GNU_SOURCE -D __STDC_CONSTANT_MACROS -D __STDC_FORMAT_MACROS -D __STDC_LIMIT_MACROS -D LLVM9 -D _FORTIFY_SOURCE=2 -D "VERSION=\"2.52b\"" -internal-isystem /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9 -internal-isystem /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/x86_64-linux-gnu/c++/9 -internal-isystem /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/x86_64-linux-gnu/c++/9 -internal-isystem /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/backward -internal-isystem /usr/local/include -internal-isystem /usr/lib/llvm-9/lib/clang/9.0.0/include -internal-externc-isystem /usr/include/x86_64-linux-gnu -internal-externc-isystem /include -internal-externc-isystem /usr/include -Wall -Wno-pointer-sign -Wno-variadic-macros -std=c++11 -fdeprecated-macro -fdebug-compilation-dir /home/adrian/Superion/llvm_mode -ferror-limit 19 -fmessage-length 0 -fno-rtti -fobjc-runtime=gcc -fdiagnostics-show-option -fcolor-diagnostics -faddrsig -o /tmp/afl-llvm-pass-30601d.o -x c++ afl-llvm-pass.so.cc clang -cc1 version 9.0.0 based upon LLVM 9.0.0 default target x86_64-pc-linux-gnu ignoring nonexistent directory "/include" ignoring duplicate directory "/usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/x86_64-linux-gnu/c++/9" #include "..." search starts here: #include <...> search starts here: /usr/lib/llvm-9/include /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9 /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/x86_64-linux-gnu/c++/9 /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/backward /usr/local/include /usr/lib/llvm-9/lib/clang/9.0.0/include /usr/include/x86_64-linux-gnu /usr/include End of search list. "/usr/bin/ld" -z relro --hash-style=gnu --build-id --eh-frame-hdr -m elf_x86_64 -shared -o ../afl-llvm-pass.so /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/crti.o /usr/bin/../lib/gcc/x86_64-linux-gnu/9/crtbeginS.o -L/usr/lib/llvm-9/lib -L/usr/bin/../lib/gcc/x86_64-linux-gnu/9 -L/usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu -L/lib/x86_64-linux-gnu -L/lib/../lib64 -L/usr/lib/x86_64-linux-gnu -L/usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../.. -L/usr/lib/llvm-9/bin/../lib -L/lib -L/usr/lib /tmp/afl-llvm-pass-30601d.o -lstdc++ -lm -lgcc_s -lgcc -lc -lc -lgcc_s -lgcc /usr/bin/../lib/gcc/x86_64-linux-gnu/9/crtendS.o /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/crtn.o clang -v -O3 -funroll-loops -v -Wall -D_FORTIFY_SOURCE=2 -g -Wno-pointer-sign -DAFL_PATH=\"/usr/local/lib/afl\" -DBIN_PATH=\"/usr/local/bin\" -DVERSION=\"2.52b\" -fPIC -c afl-llvm-rt.o.c -o ../afl-llvm-rt.o clang version 9.0.0-2 (tags/RELEASE_900/final) Target: x86_64-pc-linux-gnu Thread model: posix InstalledDir: /usr/bin Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/7 Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/7.5.0 Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/8 Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/9 Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/7 Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/7.5.0 Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/8 Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/9 Selected GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/9 Candidate multilib: .;@m64 Selected multilib: .;@m64 "/usr/lib/llvm-9/bin/clang" -cc1 -triple x86_64-pc-linux-gnu -emit-obj -disable-free -disable-llvm-verifier -discard-value-names -main-file-name afl-llvm-rt.o.c -mrelocation-model pic -pic-level 2 -mthread-model posix -fmath-errno -masm-verbose -mconstructor-aliases -munwind-tables -fuse-init-array -target-cpu x86-64 -dwarf-column-info -debug-info-kind=limited -dwarf-version=4 -debugger-tuning=gdb -momit-leaf-frame-pointer -v -v -coverage-notes-file /home/adrian/Superion/llvm_mode/../afl-llvm-rt.gcno -resource-dir /usr/lib/llvm-9/lib/clang/9.0.0 -D _FORTIFY_SOURCE=2 -D "AFL_PATH=\"/usr/local/lib/afl\"" -D "BIN_PATH=\"/usr/local/bin\"" -D "VERSION=\"2.52b\"" -internal-isystem /usr/local/include -internal-isystem /usr/lib/llvm-9/lib/clang/9.0.0/include -internal-externc-isystem /usr/include/x86_64-linux-gnu -internal-externc-isystem /include -internal-externc-isystem /usr/include -O3 -Wall -Wno-pointer-sign -fdebug-compilation-dir /home/adrian/Superion/llvm_mode -ferror-limit 19 -fmessage-length 0 -funroll-loops -fobjc-runtime=gcc -fdiagnostics-show-option -fcolor-diagnostics -vectorize-loops -vectorize-slp -faddrsig -o ../afl-llvm-rt.o -x c afl-llvm-rt.o.c clang -cc1 version 9.0.0 based upon LLVM 9.0.0 default target x86_64-pc-linux-gnu ignoring nonexistent directory "/include" #include "..." search starts here: #include <...> search starts here: /usr/local/include /usr/lib/llvm-9/lib/clang/9.0.0/include /usr/include/x86_64-linux-gnu /usr/include End of search list. [*] Building 32-bit variant of the runtime (-m32)... failed (that's fine) [*] Building 64-bit variant of the runtime (-m64)... success! [*] Testing the CC wrapper and instrumentation output... unset AFL_USE_ASAN AFL_USE_MSAN AFL_INST_RATIO; AFL_QUIET=1 AFL_PATH=. AFL_CC=clang ../afl-clang-fast -v -O3 -funroll-loops -v -Wall -D_FORTIFY_SOURCE=2 -g -Wno-pointer-sign -DAFL_PATH=\"/usr/local/lib/afl\" -DBIN_PATH=\"/usr/local/bin\" -DVERSION=\"2.52b\" ../test-instr.c -o test-instr clang version 9.0.0-2 (tags/RELEASE_900/final) Target: x86_64-pc-linux-gnu Thread model: posix InstalledDir: /usr/bin Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/7 Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/7.5.0 Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/8 Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/9 Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/7 Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/7.5.0 Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/8 Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/9 Selected GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/9 Candidate multilib: .;@m64 Selected multilib: .;@m64 "/usr/lib/llvm-9/bin/clang" -cc1 -triple x86_64-pc-linux-gnu -emit-obj -disable-free -disable-llvm-verifier -discard-value-names -main-file-name test-instr.c -mrelocation-model static -mthread-model posix -fmath-errno -masm-verbose -mconstructor-aliases -munwind-tables -fuse-init-array -target-cpu x86-64 -dwarf-column-info -debug-info-kind=limited -dwarf-version=4 -debugger-tuning=gdb -momit-leaf-frame-pointer -v -v -resource-dir /usr/lib/llvm-9/lib/clang/9.0.0 -D _FORTIFY_SOURCE=2 -D "AFL_PATH=\"/usr/local/lib/afl\"" -D "BIN_PATH=\"/usr/local/bin\"" -D "VERSION=\"2.52b\"" -D __AFL_HAVE_MANUAL_CONTROL=1 -D __AFL_COMPILER=1 -D FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION=1 -D "__AFL_LOOP(_A)=({ static volatile char *_B __attribute__((used)); _B = (char*)\"##SIG_AFL_PERSISTENT##\"; __attribute__((visibility(\"default\"))) int _L(unsigned int) __asm__(\"__afl_persistent_loop\"); _L(_A); })" -D "__AFL_INIT()=do { static volatile char *_A __attribute__((used)); _A = (char*)\"##SIG_AFL_DEFER_FORKSRV##\"; __attribute__((visibility(\"default\"))) void _I(void) __asm__(\"__afl_manual_init\"); _I(); } while (0)" -internal-isystem /usr/local/include -internal-isystem /usr/lib/llvm-9/lib/clang/9.0.0/include -internal-externc-isystem /usr/include/x86_64-linux-gnu -internal-externc-isystem /include -internal-externc-isystem /usr/include -O3 -Wall -Wno-pointer-sign -fdebug-compilation-dir /home/adrian/Superion/llvm_mode -ferror-limit 19 -fmessage-length 0 -funroll-loops -fobjc-runtime=gcc -fdiagnostics-show-option -fcolor-diagnostics -vectorize-loops -vectorize-slp -load ../afl-llvm-pass.so -faddrsig -o /tmp/test-instr-60261f.o -x c ../test-instr.c clang -cc1 version 9.0.0 based upon LLVM 9.0.0 default target x86_64-pc-linux-gnu ignoring nonexistent directory "/include" #include "..." search starts here: #include <...> search starts here: /usr/local/include /usr/lib/llvm-9/lib/clang/9.0.0/include /usr/include/x86_64-linux-gnu /usr/include End of search list. Stack dump: 0. Program arguments: /usr/lib/llvm-9/bin/clang -cc1 -triple x86_64-pc-linux-gnu -emit-obj -disable-free -disable-llvm-verifier -discard-value-names -main-file-name test-instr.c -mrelocation-model static -mthread-model posix -fmath-errno -masm-verbose -mconstructor-aliases -munwind-tables -fuse-init-array -target-cpu x86-64 -dwarf-column-info -debug-info-kind=limited -dwarf-version=4 -debugger-tuning=gdb -momit-leaf-frame-pointer -v -v -resource-dir /usr/lib/llvm-9/lib/clang/9.0.0 -D _FORTIFY_SOURCE=2 -D AFL_PATH="/usr/local/lib/afl" -D BIN_PATH="/usr/local/bin" -D VERSION="2.52b" -D __AFL_HAVE_MANUAL_CONTROL=1 -D __AFL_COMPILER=1 -D FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION=1 -D __AFL_LOOP(_A)=({ static volatile char *_B __attribute__((used)); _B = (char*)"##SIG_AFL_PERSISTENT##"; __attribute__((visibility("default"))) int _L(unsigned int) __asm__("__afl_persistent_loop"); _L(_A); }) -D __AFL_INIT()=do { static volatile char *_A __attribute__((used)); _A = (char*)"##SIG_AFL_DEFER_FORKSRV##"; __attribute__((visibility("default"))) void _I(void) __asm__("__afl_manual_init"); _I(); } while (0) -internal-isystem /usr/local/include -internal-isystem /usr/lib/llvm-9/lib/clang/9.0.0/include -internal-externc-isystem /usr/include/x86_64-linux-gnu -internal-externc-isystem /include -internal-externc-isystem /usr/include -O3 -Wall -Wno-pointer-sign -fdebug-compilation-dir /home/adrian/Superion/llvm_mode -ferror-limit 19 -fmessage-length 0 -funroll-loops -fobjc-runtime=gcc -fdiagnostics-show-option -fcolor-diagnostics -vectorize-loops -vectorize-slp -load ../afl-llvm-pass.so -faddrsig -o /tmp/test-instr-60261f.o -x c ../test-instr.c clang: error: unable to execute command: Segmentation fault (core dumped) clang: error: clang frontend command failed due to signal (use -v to see invocation) clang version 9.0.0-2 (tags/RELEASE_900/final) Target: x86_64-pc-linux-gnu Thread model: posix InstalledDir: /usr/bin clang: note: diagnostic msg: PLEASE submit a bug report to https://bugs.llvm.org/ and include the crash backtrace, preprocessed source, and associated run script. clang: error: unable to execute command: Segmentation fault (core dumped) clang: note: diagnostic msg: Error generating preprocessed source(s). make: *** [Makefile:98: test_build] Error 254

While building the ANTLR runtime on Ubuntu 16.04 with gcc-5.4.0 and clang-3.8, running the following commands:

cd /path_to_Superion/tree_mutation/
cmake ./
make

Outputs:

$ make
[  0%] Built target make_lib_output_dir
[  0%] Building CXX object runtime/CMakeFiles/antlr4_shared.dir/src/CharStream.cpp.o
c++: error: unrecognized command line option '-stdlib=libc++'
runtime/CMakeFiles/antlr4_shared.dir/build.make:62: recipe for target 'runtime/CMakeFiles/antlr4_shared.dir/src/CharStream.cpp.o' failed
make[2]: *** [runtime/CMakeFiles/antlr4_shared.dir/src/CharStream.cpp.o] Error 1
CMakeFiles/Makefile2:89: recipe for target 'runtime/CMakeFiles/antlr4_shared.dir/all' failed
make[1]: *** [runtime/CMakeFiles/antlr4_shared.dir/all] Error 2
Makefile:127: recipe for target 'all' failed
make: *** [all] Error 2

Any help would be appreciated.

Hello,

I needed support of other input grammar, so I used this command: java -jar antlr-4.7.2-complete.jar -o path -visitor -no-listener -Dlanguage=Cpp Grammar.g4. Apparently, only the GrammarSecondVisitor, which is also used in the tree mutation strategy, was not generated.

Is there other way (which is not specified in the README file) to generate the second visitor for a grammar or is this implemented manually specifically to the given grammar?

Thanks!

I don't get any output when running:

java -jar antlr-4.7-complete.jar -o path -visitor -no-listener -Dlanguage=Cpp XMLLexer.g4
java -jar antlr-4.7-complete.jar -o path -visitor -no-listener -Dlanguage=Cpp XMLParser.g4
java -jar ../xml_parser/antlr-4.7-complete.jar -o path -visitor -no-listener -Dlanguage=cpp ECMAScript.g4

Hi, I built for js_parser but why it asks for to build php_parser?

c++: error: unrecognized command line option '-stdlib=libc++'

On Docker Ubuntu:20.04,

FROM ubuntu:20.04
ENV DEBIAN_FRONTEND noninteractive
RUN apt update -y && apt install -y llvm-8 clang make git tmux wget default-jre uuid-dev libssl-dev pkg-config && \ 
wget https://github.com/Kitware/CMake/releases/download/v3.17.3/cmake-3.17.3.tar.gz && \
tar -zxvf cmake-3.17.3.tar.gz && \
cd cmake-3.17.3 && \
./bootstrap && make && make install && \
mkdir -p /fuzzers && \
cd /fuzzers && \
git clone https://github.com/zhunki/Superion.git && \
cd /fuzzers/Superion/tree_mutation/ && \
CC=clang CXX=clang++ cmake ./ && make

Firstly, Congratulations on your work, it is excellent!

And I need your help with a problem after reading the paper carefully. As you mentioned in the Grammar-Aware Trimming Strategy section of your article :

otherwise, it attempts to trim a subtree n from tree (Line 6–7). If the coverage is different after n is trimmed, then n cannot be trimmed (Line 12–14)

I wonder how to compare the coverage changes before and after trimming the test case. Which piece of code implements this functionality? I cannot find the code because of my poor programming ability.

Looking forward to your reply, Thanks in advance!