test-pipeline / orthrus Goto Github PK

A tool to manage, conduct, and assess dictionary-based fuzz testing

License: GNU General Public License v3.0

Python 99.06% Shell 0.94%

orthrus afl-fuzz dictionary

orthrus's Introduction

Orthrus

Orthrus is a tool for managing, conducting, and assessing dictionary-based security (fuzz) testing for autotools projects. At the moment, it supports Clang/LLVM instrumentation and the AFL ecosystem (afl-fuzz, afl-utils, afl-cov). The ultimate aim is for Orthrus to be a generic wrapper around state-of-the-art fuzz and instrumentation tools on the one hand, and disparate build systems on the other.

NEW: The dictionary-based fuzzing feature is new. Do orthrus create -dict to generate a fuzzing dictionary and orthrus add --jobconf to specify fuzz options (e.g., -x dict) for making use of the generated dictionary for fuzzing.

Installation

Please read docs/Getting_started.md.

Workflow

Orthrus currently supports two workflows. In a routine workflow, you work with a single fuzzing job end-to-end i.e., from source code instrumentation, until crash triage. In a A/B test workflow, you work with a single A/B test end-to-end.

Routine

Please read docs/Workflow.md.

A/B testing

Please read docs/Workflow_abtests.md.

Full usage

$ orthrus -h
usage: Orthrus 1.1 by Bhargava Shastry, and Markus Leutner <https://github.com/test-pipeline/orthrus> 
       [-h] [-v]
       {create,add,remove,start,stop,show,triage,coverage,spectrum,runtime,destroy,validate}
       ...

optional arguments:
  -h, --help            show this help message and exit
  -v, --verbose         Verbose mode, print information about the progress

subcommands:
  Orthrus subcommands

  {create,add,remove,start,stop,show,triage,coverage,spectrum,runtime,destroy,validate}
    create              Create an orthrus workspace
    add                 Add a fuzzing job
    remove              Remove a fuzzing job
    start               Start a fuzzing jobs
    stop                Stop a fuzzing jobs
    show                Show what's currently going on
    triage              Triage crash corpus
    coverage            Run afl-cov on existing AFL corpus
    spectrum            Run spectrum based analysis on existing AFL corpus
    runtime             Perform dynamic analysis of existing AFL corpus
    destroy             Destroy an orthrus workspace
    validate            Check if all Orthrus dependencies are met

Issues and PRs

Feel free to file an issue if something doesn't work as expected :-)
- Attaching logs from .orthrus/logs would be helpful
PRs for interesting workflows are much appreciated!

Credits

Orthrus was possible due to excellent work by

lcamtuf (afl-fuzz)
rc0r (afl-utils)
Michael Rash (afl-cov)
Clang/LLVM sanitization projects
Folks at afl users community and beyond

orthrus's People

Contributors

Stargazers

Watchers

Forkers

gitcollect cys3c zined chubbymaggie terry2012 fuzzamos ufwt fatgrass jingqiangliu sthagen mr-segfault daydayup40 5l1v3r1 bathepawan

orthrus's Issues

Workaround for afl-utils 'interactive' bug

Bug in afl-utils v1.32 (https://github.com/rc0r/afl-utils/issues/34#issuecomment-313678837) leads to process group ids not being written to disk and fuzz processes not getting killed on afl-multikill. Workaround this bug by explicitly defining 'interactive' to false (0).

orthrus create failed for libpcap

Hi
I followed your tutorial to setup orthrus with success but when I wanna create the binaries for my libpcap fuzz session I get the following stacktrace:

~/workspace/fuzzing/pcap/libpcap   master  orthrus create -fuzz -asan -cov
Orthrus 1.01 by Markus Leutner, and Bhargava Shastry <https://github.com/test-pipeline/orthrus> 

[+] Create Orthrus workspace
    [+] Installing binaries for afl-fuzz with AddressSanitizer
        [+] Configure... done
        [+] Compile and install... done
Traceback (most recent call last):
  File "/home/afl/.virtualenvs/afl/bin/orthrus", line 4, in <module>
    __import__('pkg_resources').run_script('orthrus===0.1.-pre-alpha-', 'orthrus')
  File "/home/afl/.virtualenvs/afl/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 743, in run_script
    self.require(requires)[0].run_script(script_name, ns)
  File "/home/afl/.virtualenvs/afl/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 1531, in run_script
    exec(code, namespace, namespace)
  File "/home/afl/.virtualenvs/afl/lib/python2.7/site-packages/orthrus-0.1._pre_alpha_-py2.7.egg/EGG-INFO/scripts/orthrus", line 67, in <module>
    tool.run()
  File "/home/afl/.virtualenvs/afl/lib/python2.7/site-packages/orthrus-0.1._pre_alpha_-py2.7.egg/EGG-INFO/scripts/orthrus", line 63, in run
    return self._args.func(self._args)
  File "/home/afl/.virtualenvs/afl/lib/python2.7/site-packages/orthrus-0.1._pre_alpha_-py2.7.egg/EGG-INFO/scripts/orthrus", line 22, in _create
    cmd.run()
  File "/home/afl/.virtualenvs/afl/local/lib/python2.7/site-packages/orthrus-0.1._pre_alpha_-py2.7.egg/orthrus/commands.py", line 127, in run
    if not self.create(install_path, b.BuildEnv.BEnv_afl_asan, 'afl-asan_inst.log'):
  File "/home/afl/.virtualenvs/afl/local/lib/python2.7/site-packages/orthrus-0.1._pre_alpha_-py2.7.egg/orthrus/commands.py", line 95, in create
    sample_binpath = random.choice(util.return_elf_binaries(install_path + 'bin/'))
  File "/usr/lib/python2.7/random.py", line 275, in choice
    return seq[int(self.random() * len(seq))]  # raises IndexError if seq is empty
IndexError: list index out of range

Any idea on what I am doing wrong?

My setup is a debian jessie box up-to-date, running orthrus and afl-utils in a virtualenv.

Thanks

PS: here is the afl-asan_inst.log where I cannot find anything wrong:

rm -f pcap-linux.o pcap-usb-linux.o pcap-bt-linux.o pcap-bt-monitor-linux.o pcap-netfilter-linux.o pcap-dbus.o fad-getad.o pcap.o inet.o fad-helpers.o gencode.o optimize.o nametoaddr.o etherent.o savefile.o sf-pcap.o sf-pcap-ng.o pcap-common.o bpf_image.o bpf_dump.o  scanner.o grammar.o bpf_filter.o version.o  libpcap.* valgrindtest capturetest can_set_rfmon_test filtertest findalldevstest opentest selpolltest libpcap-`cat ./VERSION`.tar.gz scanner.c grammar.c bpf_filter.c version.c scanner.h grammar.h pcap_version.h lex.yy.c pcap-config
afl-clang -fvisibility=hidden -fpic -O3 -I.  -I/usr/include/dbus-1.0 -I/usr/lib/x86_64-linux-gnu/dbus-1.0/include  -DBUILDING_PCAP -DHAVE_CONFIG_H  -D_U_="__attribute__((unused))" -O3    -c ./pcap-linux.c
afl-clang -fvisibility=hidden -fpic -O3 -I.  -I/usr/include/dbus-1.0 -I/usr/lib/x86_64-linux-gnu/dbus-1.0/include  -DBUILDING_PCAP -DHAVE_CONFIG_H  -D_U_="__attribute__((unused))" -O3    -c ./pcap-usb-linux.c
afl-clang -fvisibility=hidden -fpic -O3 -I.  -I/usr/include/dbus-1.0 -I/usr/lib/x86_64-linux-gnu/dbus-1.0/include  -DBUILDING_PCAP -DHAVE_CONFIG_H  -D_U_="__attribute__((unused))" -O3    -c ./pcap-bt-linux.c
afl-clang -fvisibility=hidden -fpic -O3 -I.  -I/usr/include/dbus-1.0 -I/usr/lib/x86_64-linux-gnu/dbus-1.0/include  -DBUILDING_PCAP -DHAVE_CONFIG_H  -D_U_="__attribute__((unused))" -O3    -c ./pcap-bt-monitor-linux.c
afl-clang -fvisibility=hidden -fpic -O3 -I.  -I/usr/include/dbus-1.0 -I/usr/lib/x86_64-linux-gnu/dbus-1.0/include  -DBUILDING_PCAP -DHAVE_CONFIG_H  -D_U_="__attribute__((unused))" -O3    -c ./pcap-netfilter-linux.c
afl-clang -fvisibility=hidden -fpic -O3 -I.  -I/usr/include/dbus-1.0 -I/usr/lib/x86_64-linux-gnu/dbus-1.0/include  -DBUILDING_PCAP -DHAVE_CONFIG_H  -D_U_="__attribute__((unused))" -O3    -c ./pcap-dbus.c
afl-clang -fvisibility=hidden -fpic -O3 -I.  -I/usr/include/dbus-1.0 -I/usr/lib/x86_64-linux-gnu/dbus-1.0/include  -DBUILDING_PCAP -DHAVE_CONFIG_H  -D_U_="__attribute__((unused))" -O3    -c ./fad-getad.c
bison -y -p pcap_ -o grammar.c -d grammar.y
./gen_version_header.sh ./VERSION ./pcap_version.h.in pcap_version.h
flex -P pcap_ --header-file=scanner.h --nounput -o scanner.c scanner.l
afl-clang -fvisibility=hidden -fpic -O3 -I.  -I/usr/include/dbus-1.0 -I/usr/lib/x86_64-linux-gnu/dbus-1.0/include  -DBUILDING_PCAP -DHAVE_CONFIG_H  -D_U_="__attribute__((unused))" -O3    -c ./inet.c
afl-clang -fvisibility=hidden -fpic -O3 -I.  -I/usr/include/dbus-1.0 -I/usr/lib/x86_64-linux-gnu/dbus-1.0/include  -DBUILDING_PCAP -DHAVE_CONFIG_H  -D_U_="__attribute__((unused))" -O3    -c ./fad-helpers.c
afl-clang -fvisibility=hidden -fpic -O3 -I.  -I/usr/include/dbus-1.0 -I/usr/lib/x86_64-linux-gnu/dbus-1.0/include  -DBUILDING_PCAP -DHAVE_CONFIG_H  -D_U_="__attribute__((unused))" -O3    -c ./nametoaddr.c
afl-clang -fvisibility=hidden -fpic -O3 -I.  -I/usr/include/dbus-1.0 -I/usr/lib/x86_64-linux-gnu/dbus-1.0/include  -DBUILDING_PCAP -DHAVE_CONFIG_H  -D_U_="__attribute__((unused))" -O3    -c ./etherent.c
afl-clang -fvisibility=hidden -fpic -O3 -I.  -I/usr/include/dbus-1.0 -I/usr/lib/x86_64-linux-gnu/dbus-1.0/include  -DBUILDING_PCAP -DHAVE_CONFIG_H  -D_U_="__attribute__((unused))" -O3    -c ./savefile.c
afl-clang -fvisibility=hidden -fpic -O3 -I.  -I/usr/include/dbus-1.0 -I/usr/lib/x86_64-linux-gnu/dbus-1.0/include  -DBUILDING_PCAP -DHAVE_CONFIG_H  -D_U_="__attribute__((unused))" -O3    -c ./sf-pcap.c
afl-clang -fvisibility=hidden -fpic -O3 -I.  -I/usr/include/dbus-1.0 -I/usr/lib/x86_64-linux-gnu/dbus-1.0/include  -DBUILDING_PCAP -DHAVE_CONFIG_H  -D_U_="__attribute__((unused))" -O3    -c ./sf-pcap-ng.c
rm -f bpf_filter.c
#
afl-clang -fvisibility=hidden -fpic -O3 -I.  -I/usr/include/dbus-1.0 -I/usr/lib/x86_64-linux-gnu/dbus-1.0/include  -DBUILDING_PCAP -DHAVE_CONFIG_H  -D_U_="__attribute__((unused))" -O3    -c ./bpf_image.c
afl-clang -fvisibility=hidden -fpic -O3 -I.  -I/usr/include/dbus-1.0 -I/usr/lib/x86_64-linux-gnu/dbus-1.0/include  -DBUILDING_PCAP -DHAVE_CONFIG_H  -D_U_="__attribute__((unused))" -O3    -c ./bpf_dump.c
ln -s ./bpf/net/bpf_filter.c bpf_filter.c
./config.status --file=pcap-config.tmp:./pcap-config.in
afl-clang -fvisibility=hidden -fpic -O3 -I.  -I/usr/include/dbus-1.0 -I/usr/lib/x86_64-linux-gnu/dbus-1.0/include  -DBUILDING_PCAP -DHAVE_CONFIG_H  -D_U_="__attribute__((unused))" -O3    -c bpf_filter.c
# Older programs import this if they want to show the
# libpcap version number, rather than calling
afl-clang -fvisibility=hidden -fpic -O3 -I.  -I/usr/include/dbus-1.0 -I/usr/lib/x86_64-linux-gnu/dbus-1.0/include  -DBUILDING_PCAP -DHAVE_CONFIG_H  -D_U_="__attribute__((unused))" -O3    -c ./pcap-common.c
afl-clang -fvisibility=hidden -fpic -O3 -I.  -I/usr/include/dbus-1.0 -I/usr/lib/x86_64-linux-gnu/dbus-1.0/include  -DBUILDING_PCAP -DHAVE_CONFIG_H  -D_U_="__attribute__((unused))" -O3    -c ./optimize.c
grammar.y: warning: 38 shift/reduce conflicts [-Wconflicts-sr]
# pcap_lib_version(), so we need to export it.
#
./gen_version_c.sh ./VERSION version.c
afl-clang -fvisibility=hidden -fpic -O3 -I.  -I/usr/include/dbus-1.0 -I/usr/lib/x86_64-linux-gnu/dbus-1.0/include  -DBUILDING_PCAP -DHAVE_CONFIG_H  -D_U_="__attribute__((unused))" -O3    -c ./pcap.c
afl-clang -fvisibility=hidden -fpic -O3 -I.  -I/usr/include/dbus-1.0 -I/usr/lib/x86_64-linux-gnu/dbus-1.0/include  -DBUILDING_PCAP -DHAVE_CONFIG_H  -D_U_="__attribute__((unused))" -O3    -c version.c
config.status: creating pcap-config.tmp
mv pcap-config.tmp pcap-config
chmod a+x pcap-config
afl-clang -fvisibility=hidden -fpic -O3 -I.  -I/usr/include/dbus-1.0 -I/usr/lib/x86_64-linux-gnu/dbus-1.0/include  -DBUILDING_PCAP -DHAVE_CONFIG_H  -D_U_="__attribute__((unused))" -O3    -c grammar.c
afl-clang -fvisibility=hidden -fpic -O3 -I.  -I/usr/include/dbus-1.0 -I/usr/lib/x86_64-linux-gnu/dbus-1.0/include  -DBUILDING_PCAP -DHAVE_CONFIG_H  -D_U_="__attribute__((unused))" -O3    -c ./gencode.c
afl-clang -fvisibility=hidden -fpic -O3 -I.  -I/usr/include/dbus-1.0 -I/usr/lib/x86_64-linux-gnu/dbus-1.0/include  -DBUILDING_PCAP -DHAVE_CONFIG_H  -D_U_="__attribute__((unused))" -O3    -c scanner.c
VER=`cat ./VERSION`; \
MAJOR_VER=`sed 's/\([0-9][0-9]*\)\..*/\1/' ./VERSION`; \
afl-clang -shared -Wl,-soname,libpcap.so.$MAJOR_VER  \
    -o libpcap.so.$VER pcap-linux.o pcap-usb-linux.o pcap-bt-linux.o pcap-bt-monitor-linux.o pcap-netfilter-linux.o pcap-dbus.o fad-getad.o pcap.o inet.o fad-helpers.o gencode.o optimize.o nametoaddr.o etherent.o savefile.o sf-pcap.o sf-pcap-ng.o pcap-common.o bpf_image.o bpf_dump.o  scanner.o grammar.o bpf_filter.o version.o   -ldbus-1 
ar rc libpcap.a pcap-linux.o pcap-usb-linux.o pcap-bt-linux.o pcap-bt-monitor-linux.o pcap-netfilter-linux.o pcap-dbus.o fad-getad.o pcap.o inet.o fad-helpers.o gencode.o optimize.o nametoaddr.o etherent.o savefile.o sf-pcap.o sf-pcap-ng.o pcap-common.o bpf_image.o bpf_dump.o  scanner.o grammar.o bpf_filter.o version.o  
ranlib libpcap.a
#
# Most platforms have separate suffixes for shared and
# archive libraries, so we install both.
#
[ -d /home/afl/workspace/fuzzing/pcap/libpcap/.orthrus/binaries/afl-asan/lib ] || \
    (mkdir -p /home/afl/workspace/fuzzing/pcap/libpcap/.orthrus/binaries/afl-asan/lib; chmod 755 /home/afl/workspace/fuzzing/pcap/libpcap/.orthrus/binaries/afl-asan/lib)
/usr/bin/install -c -m 644 libpcap.a /home/afl/workspace/fuzzing/pcap/libpcap/.orthrus/binaries/afl-asan/lib/libpcap.a
ranlib /home/afl/workspace/fuzzing/pcap/libpcap/.orthrus/binaries/afl-asan/lib/libpcap.a
[ -d /home/afl/workspace/fuzzing/pcap/libpcap/.orthrus/binaries/afl-asan/lib ] || \
    (mkdir -p /home/afl/workspace/fuzzing/pcap/libpcap/.orthrus/binaries/afl-asan/lib; chmod 755 /home/afl/workspace/fuzzing/pcap/libpcap/.orthrus/binaries/afl-asan/lib)
VER=`cat ./VERSION`; \
MAJOR_VER=`sed 's/\([0-9][0-9]*\)\..*/\1/' ./VERSION`; \
/usr/bin/install -c libpcap.so.$VER /home/afl/workspace/fuzzing/pcap/libpcap/.orthrus/binaries/afl-asan/lib/libpcap.so.$VER; \
ln -sf libpcap.so.$VER /home/afl/workspace/fuzzing/pcap/libpcap/.orthrus/binaries/afl-asan/lib/libpcap.so.$MAJOR_VER; \
ln -sf libpcap.so.$MAJOR_VER /home/afl/workspace/fuzzing/pcap/libpcap/.orthrus/binaries/afl-asan/lib/libpcap.so
[ -d /home/afl/workspace/fuzzing/pcap/libpcap/.orthrus/binaries/afl-asan/lib ] || \
    (mkdir -p /home/afl/workspace/fuzzing/pcap/libpcap/.orthrus/binaries/afl-asan/lib; chmod 755 /home/afl/workspace/fuzzing/pcap/libpcap/.orthrus/binaries/afl-asan/lib)
[ -d /home/afl/workspace/fuzzing/pcap/libpcap/.orthrus/binaries/afl-asan/include ] || \
    (mkdir -p /home/afl/workspace/fuzzing/pcap/libpcap/.orthrus/binaries/afl-asan/include; chmod 755 /home/afl/workspace/fuzzing/pcap/libpcap/.orthrus/binaries/afl-asan/include)
[ -d /home/afl/workspace/fuzzing/pcap/libpcap/.orthrus/binaries/afl-asan/include/pcap ] || \
    (mkdir -p /home/afl/workspace/fuzzing/pcap/libpcap/.orthrus/binaries/afl-asan/include/pcap; chmod 755 /home/afl/workspace/fuzzing/pcap/libpcap/.orthrus/binaries/afl-asan/include/pcap)
[ -d /home/afl/workspace/fuzzing/pcap/libpcap/.orthrus/binaries/afl-asan/share/man/man1 ] || \
    (mkdir -p /home/afl/workspace/fuzzing/pcap/libpcap/.orthrus/binaries/afl-asan/share/man/man1; chmod 755 /home/afl/workspace/fuzzing/pcap/libpcap/.orthrus/binaries/afl-asan/share/man/man1)
[ -d /home/afl/workspace/fuzzing/pcap/libpcap/.orthrus/binaries/afl-asan/share/man/man3 ] || \
    (mkdir -p /home/afl/workspace/fuzzing/pcap/libpcap/.orthrus/binaries/afl-asan/share/man/man3; chmod 755 /home/afl/workspace/fuzzing/pcap/libpcap/.orthrus/binaries/afl-asan/share/man/man3)
[ -d /home/afl/workspace/fuzzing/pcap/libpcap/.orthrus/binaries/afl-asan/share/man/man5 ] || \
    (mkdir -p /home/afl/workspace/fuzzing/pcap/libpcap/.orthrus/binaries/afl-asan/share/man/man5; chmod 755 /home/afl/workspace/fuzzing/pcap/libpcap/.orthrus/binaries/afl-asan/share/man/man5)
[ -d /home/afl/workspace/fuzzing/pcap/libpcap/.orthrus/binaries/afl-asan/share/man/man7 ] || \
    (mkdir -p /home/afl/workspace/fuzzing/pcap/libpcap/.orthrus/binaries/afl-asan/share/man/man7; chmod 755 /home/afl/workspace/fuzzing/pcap/libpcap/.orthrus/binaries/afl-asan/share/man/man7)
for i in pcap.h pcap-bpf.h pcap-namedb.h pcap/bpf.h pcap/bluetooth.h pcap/can_socketcan.h pcap/dlt.h pcap/export-defs.h pcap/ipnet.h pcap/namedb.h pcap/nflog.h pcap/pcap.h pcap/sll.h pcap/vlan.h pcap/usb.h; do \
    /usr/bin/install -c -m 644 ./$i \
        /home/afl/workspace/fuzzing/pcap/libpcap/.orthrus/binaries/afl-asan/include/$i; done
[ -d /home/afl/workspace/fuzzing/pcap/libpcap/.orthrus/binaries/afl-asan/bin ] || \
    (mkdir -p /home/afl/workspace/fuzzing/pcap/libpcap/.orthrus/binaries/afl-asan/bin; chmod 755 /home/afl/workspace/fuzzing/pcap/libpcap/.orthrus/binaries/afl-asan/bin)
/usr/bin/install -c pcap-config /home/afl/workspace/fuzzing/pcap/libpcap/.orthrus/binaries/afl-asan/bin/pcap-config
for i in pcap-config.1; do \
    /usr/bin/install -c -m 644 ./$i \
        /home/afl/workspace/fuzzing/pcap/libpcap/.orthrus/binaries/afl-asan/share/man/man1/$i; done
for i in pcap_activate.3pcap pcap_breakloop.3pcap pcap_can_set_rfmon.3pcap pcap_close.3pcap pcap_create.3pcap pcap_datalink_name_to_val.3pcap pcap_datalink_val_to_name.3pcap pcap_dump.3pcap pcap_dump_close.3pcap pcap_dump_file.3pcap pcap_dump_flush.3pcap pcap_dump_ftell.3pcap pcap_file.3pcap pcap_fileno.3pcap pcap_findalldevs.3pcap pcap_freecode.3pcap pcap_get_selectable_fd.3pcap pcap_geterr.3pcap pcap_inject.3pcap pcap_is_swapped.3pcap pcap_lib_version.3pcap pcap_lookupdev.3pcap pcap_lookupnet.3pcap pcap_loop.3pcap pcap_major_version.3pcap pcap_next_ex.3pcap pcap_offline_filter.3pcap pcap_open_live.3pcap pcap_set_buffer_size.3pcap pcap_set_datalink.3pcap pcap_set_immediate_mode.3pcap pcap_set_promisc.3pcap pcap_set_rfmon.3pcap pcap_set_snaplen.3pcap pcap_set_timeout.3pcap pcap_setdirection.3pcap pcap_setfilter.3pcap pcap_setnonblock.3pcap pcap_snapshot.3pcap pcap_stats.3pcap pcap_statustostr.3pcap pcap_strerror.3pcap pcap_tstamp_type_name_to_val.3pcap pcap_tstamp_type_val_to_name.3pcap; do \
    /usr/bin/install -c -m 644 ./$i \
        /home/afl/workspace/fuzzing/pcap/libpcap/.orthrus/binaries/afl-asan/share/man/man3/$i; done
for i in pcap.3pcap pcap_compile.3pcap pcap_datalink.3pcap pcap_dump_open.3pcap pcap_get_tstamp_precision.3pcap pcap_list_datalinks.3pcap pcap_list_tstamp_types.3pcap pcap_open_dead.3pcap pcap_open_offline.3pcap pcap_set_tstamp_precision.3pcap pcap_set_tstamp_type.3pcap; do \
    /usr/bin/install -c -m 644 $i \
        /home/afl/workspace/fuzzing/pcap/libpcap/.orthrus/binaries/afl-asan/share/man/man3/$i; done
(cd /home/afl/workspace/fuzzing/pcap/libpcap/.orthrus/binaries/afl-asan/share/man/man3 && \
rm -f pcap_datalink_val_to_description.3pcap && \
ln -s pcap_datalink_val_to_name.3pcap \
     pcap_datalink_val_to_description.3pcap && \
rm -f pcap_dump_fopen.3pcap && \
ln -s pcap_dump_open.3pcap pcap_dump_fopen.3pcap && \
rm -f pcap_freealldevs.3pcap && \
ln -s pcap_findalldevs.3pcap pcap_freealldevs.3pcap && \
rm -f pcap_perror.3pcap && \
ln -s pcap_geterr.3pcap pcap_perror.3pcap && \
rm -f pcap_sendpacket.3pcap && \
ln -s pcap_inject.3pcap pcap_sendpacket.3pcap && \
rm -f pcap_free_datalinks.3pcap && \
ln -s pcap_list_datalinks.3pcap pcap_free_datalinks.3pcap && \
rm -f pcap_free_tstamp_types.3pcap && \
ln -s pcap_list_tstamp_types.3pcap pcap_free_tstamp_types.3pcap && \
rm -f pcap_dispatch.3pcap && \
ln -s pcap_loop.3pcap pcap_dispatch.3pcap && \
rm -f pcap_minor_version.3pcap && \
ln -s pcap_major_version.3pcap pcap_minor_version.3pcap && \
rm -f pcap_next.3pcap && \
ln -s pcap_next_ex.3pcap pcap_next.3pcap && \
rm -f pcap_open_dead_with_tstamp_precision.3pcap && \
ln -s pcap_open_dead.3pcap \
     pcap_open_dead_with_tstamp_precision.3pcap && \
rm -f pcap_open_offline_with_tstamp_precision.3pcap && \
ln -s pcap_open_offline.3pcap pcap_open_offline_with_tstamp_precision.3pcap && \
rm -f pcap_fopen_offline.3pcap && \
ln -s pcap_open_offline.3pcap pcap_fopen_offline.3pcap && \
rm -f pcap_fopen_offline_with_tstamp_precision.3pcap && \
ln -s pcap_open_offline.3pcap pcap_fopen_offline_with_tstamp_precision.3pcap && \
rm -f pcap_tstamp_type_val_to_description.3pcap && \
ln -s pcap_tstamp_type_val_to_name.3pcap pcap_tstamp_type_val_to_description.3pcap && \
rm -f pcap_getnonblock.3pcap && \
ln -s pcap_setnonblock.3pcap pcap_getnonblock.3pcap)
for i in pcap-savefile.manfile.in; do \
    /usr/bin/install -c -m 644 `echo $i | sed 's/.manfile.in/.manfile/'` \
        /home/afl/workspace/fuzzing/pcap/libpcap/.orthrus/binaries/afl-asan/share/man/man5/`echo $i | sed 's/.manfile.in/.5/'`; done
for i in pcap-filter.manmisc.in pcap-linktype.manmisc.in pcap-tstamp.manmisc.in; do \
    /usr/bin/install -c -m 644 `echo $i | sed 's/.manmisc.in/.manmisc/'` \
        /home/afl/workspace/fuzzing/pcap/libpcap/.orthrus/binaries/afl-asan/share/man/man7/`echo $i | sed 's/.manmisc.in/.7/'`; done

Archive runtime results if it exists

Use a --regenerate flag for orthrus runtime to do as subject line says.

[Add/Start/Stop] Implement support for afl fuzz in QEMU mode

This takes issue #13 one step forward towards supporting binary only fuzzing jobs. Note that this increases afl-utils dependency to [v1.33a][https://github.com/rc0r/afl-utils/releases/tag/v1.33a]

Issue closes if:

job configuration exposes an option called qemu (that accepts a boolean value) that ensures that job is fuzzed in afl qemu mode (-Q)
corpus minimization is supported (requires afl-utils v1.33a)

missing --coverage in gcc_coverage build environment

I'm trying with mp3gain, this program do not provide configure, so touch configure && chmod +x configure first.

# orthrus create -cov
Orthrus 1.2 by Bhargava Shastry, and Markus Leutner <https://github.com/test-pipeline/orthrus>

[+] Creating Orthrus workspace
                [+] Checking if workspace exists... done
                [+] Configuring... done
                [+] Compiling... failed
        [+] Installing binaries for obtaining test coverage information... failed


cat .orthrus/logs/gcc_coverage.log
rm -rf mp3gain mp3gain.zip mp3gain.o apetag.o id3tag.o gain_analysis.o rg_error.o
gcc -g -O0 -fprofile-arcs -ftest-coverage -Wall -DHAVE_MEMCPY   -c -o mp3gain.o mp3gain.c
gcc -g -O0 -fprofile-arcs -ftest-coverage -Wall -DHAVE_MEMCPY   -c -o apetag.o apetag.c
gcc -g -O0 -fprofile-arcs -ftest-coverage -Wall -DHAVE_MEMCPY   -c -o id3tag.o id3tag.c
gcc -g -O0 -fprofile-arcs -ftest-coverage -Wall -DHAVE_MEMCPY   -c -o gain_analysis.o gain_analysis.c
gcc -g -O0 -fprofile-arcs -ftest-coverage -Wall -DHAVE_MEMCPY   -c -o rg_error.o rg_error.c
mp3gain.c: In function ‘changeGain’:
mp3gain.c:702:7: warning: variable ‘freqidx’ set but not used [-Wunused-but-set-variable]
   int freqidx;
       ^
mp3gain.c: In function ‘main’:
mp3gain.c:1465:6: warning: variable ‘crcflag’ set but not used [-Wunused-but-set-variable]
  int crcflag;
      ^
apetag.c: In function ‘ReadMP3APETag’:
apetag.c:165:33: warning: variable ‘curFieldNum’ set but not used [-Wunused-but-set-variable]
     unsigned long               curFieldNum;
                                 ^
apetag.c:154:33: warning: variable ‘flags’ set but not used [-Wunused-but-set-variable]
     unsigned long               flags;
                                 ^
gcc -lgcov -o mp3gain mp3gain.o apetag.o id3tag.o gain_analysis.o rg_error.o  -lm -lmpg123
mp3gain.o: In function `_GLOBAL__sub_I_65535_0_writeself':
/work/output/orthrus/mp3gain.c:2764: undefined reference to `__gcov_init'
mp3gain.o:(.data+0x80): undefined reference to `__gcov_merge_add'
apetag.o: In function `_GLOBAL__sub_I_65535_0_ReadMP3ID3v1Tag':
/work/output/orthrus/apetag.c:694: undefined reference to `__gcov_init'
apetag.o:(.data+0x60): undefined reference to `__gcov_merge_add'
id3tag.o: In function `_GLOBAL__sub_I_65535_0_ReadMP3GainID3Tag':
/work/output/orthrus/id3tag.c:1417: undefined reference to `__gcov_init'
id3tag.o:(.data+0x60): undefined reference to `__gcov_merge_add'
gain_analysis.o: In function `_GLOBAL__sub_I_65535_0_ResetSampleFrequency':
/work/output/orthrus/gain_analysis.c:479: undefined reference to `__gcov_init'
gain_analysis.o:(.data+0x60): undefined reference to `__gcov_merge_add'
rg_error.o: In function `_GLOBAL__sub_I_65535_0_DoError':
/work/output/orthrus/rg_error.c:63: undefined reference to `__gcov_init'
rg_error.o:(.data+0x60): undefined reference to `__gcov_merge_add'
collect2: error: ld returned 1 exit status
Makefile:55: recipe for target 'mp3gain' failed
make: *** [mp3gain] Error 1

And I found: https://stackoverflow.com/questions/16682606/how-to-resovle-gcov-init-undefined-reference-issue-when-link

which says LFLAGS: -lgcov --coverage, so I make this patch before python setup.py install and it works:

sed -i "s#'-lgcov'#'-lgcov --coverage'#g" orthrus/builder/builder.py

related code:

orthrus/builder/builder.py

Lines 46 to 47 in 7e916f3

    
           BEnv_gcc_coverage = BEnv('gcc', 'g++', '-g -O0 -fprofile-arcs -ftest-coverage', 
        
                               '-g -O0 -fprofile-arcs -ftest-coverage', '-lgcov', '-lgcov', {})

Python 3 support

Python 2 is almost dead, only a few years of agony are left for it.

Implement afl dict auto create during create and use during add job

Something like `orthrus create ... -autodict
- Invokes libtooling under the hood
During add job, something like orthrus add ... -dict=sample.dict

Archive binaries instead of prompting for a destroy

At the moment, there is no clean way to resume fuzzing after modifying source code. Users are prompted to do a full destroy via orthrus destroy.

Archive option during create will help archive old binaries dir before creating a new one.

[Show] Expose seeds and qemu in orthrus show

Issue closes if

orthrus show -conf shows seeds dir and qemu mode for configured jobs.

Refactor code in commands into separate files

Issue closes if:

there is one py file per orthrus command

[Add] Fold add options into jobconf

Issue closes if

orthrus add --jobconf=test.conf is all that is neeeded to add a fuzzing job
- job desc, job type, seed dir are to be defined in test.conf
- Bonus points for exposing num cores to utilize a la afl-multicore

Bug: Stopping job in ASAN only mode does not terminate fuzzing sessions

Why [+] Configuring... failed. Do Create instrumented binaries run under source package files?

(afl) blu3sh0rk@orthrus:~/Desktop/orthrus/testdata/libtiff$ orthrus create -fuzz -asan -cov
Orthrus 1.2 by Bhargava Shastry, and Markus Leutner <https://github.com/test-pipeline/orthrus> 

[+] Creating Orthrus workspace
                [+] Checking if workspace exists... done
                [+] Configuring... failed
        [+] Installing binaries for afl-fuzz with AddressSanitizer... failed
(afl) blu3sh0rk@orthrus:~/Desktop/orthrus/testdata/libtiff$ ls
aclocal.m4  autogen.sh      build      cmake           COMMITTERS  configure     contrib          doc            HOWTO-SECURITY-RELEASE  libtiff-4.pc.in  m4           placeholder.h  README.md     test       TODO   VERSION
archive     autom4te.cache  ChangeLog  CMakeLists.txt  config.log  configure.ac  CONTRIBUTING.md  HOWTO-RELEASE  libtiff                 LICENSE.md       Makefile.am  port           RELEASE-DATE  tiff.spec  tools

Do Create instrumented binaries run under source package files ? like ~/Desktop/orthrus/testdata/libtiff

In A/B test mode with corpus minimization, start both fuzzers almost simultaneously

At the moment, we do

Minimize corpus for Fuzzer A
Start fuzzer A
Minimize corpus for Fuzzer B
Start fuzzer B

The drawback of this workflow is that the time required to minimize corpus might be quite high due to which the second fuzzer is not started exactly after starting fuzzer A.

Rework the workflow, like so:

Minimize corpus for Fuzzer A
Minimize corpus for Fuzzer B
Start fuzzer A
Start fuzzer B

[Create] Implement create feature for binary only fuzzing

When a project is not OS, we can still fuzz it using afl-fuzz qemu support. For this, we need orthrus create to implement boilerplate code to organize binaries in a project.

Issue closes if:

orthrus create -binary or some such
- places executable binaries in .orthrus/binaries/afl-qemu/bin

Fail elegantly when AFL_HARDEN=1 poses a compilation issue

AFL_HARDEN=1 env variable / -DFORTIFY.. compilation flag may sometimes cause a compilation error. It would be nice to retry minus those settings when the failure happens instead of bailing out. It's better to have some afl binary than none at all.

Issue closes if
- orthrus runtime supports a/b test jobs, and
- orthrus spectrum supports a/b test jobs

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.

	BEnv_gcc_coverage = BEnv('gcc', 'g++', '-g -O0 -fprofile-arcs -ftest-coverage',
	'-g -O0 -fprofile-arcs -ftest-coverage', '-lgcov', '-lgcov', {})