See title.

When afl starts, it first confirms if the command it's about to fuzz even makes sense by running the test case. afl-cov should do something similar.

I had an issue where due to some error on my side, the instance that was being run to get the coverage failed even though the afl ran instance succeeded. Took me a long time to realize this :)

If I understood the docs correctly, --disable-coverage-init is supposed to allow updating already existing coverage information with new test cases. However, I was unable to make it work, since it either complains that an existing cov directory was found or, if told to overwrite the directory with -O, it says /tmp/afl-ramdisk/out/cov/lcov/trace.lcov_info_final.

Could you add an example of this flag's usage to the README?

Hi, I use aflnet to test a network program. When I finished test, I want to use afl-cov to see the coverage of the program. But afl-cov use stdin or file as input. However aflnet's testcase is network package.
Can you help me? Thank you very much.

Hello,

I am running the newest version of afl-cov and afl on 64-bit Ubuntu 14.04. I followed the tutorial here, and I am able to successfully fuzz a test code with afl.

However, when I tried to run afl-cov, I got the following error:

☁  cov  afl-cov -d ../afl_out --live --coverage-cmd "./checksum AFL_FILE" --code-dir ../ --overwrite
    Non-zero exit status '255' for CMD: /usr/bin/lcov --no-checksum --capture --initial --directory ../ --output-file ../afl_out/cov/lcov/trace.lcov_base

*** Imported 3 new test cases from: ../afl_out/queue

[+] AFL test case: id:000000,orig:in (0 / 3), cycle: 0

    Non-zero exit status '255' for CMD: /usr/bin/lcov --no-checksum --capture --directory ../ --output-file ../afl_out/cov/lcov/trace.lcov_info
    Non-zero exit status '255' for CMD: /usr/bin/lcov --no-checksum -a ../afl_out/cov/lcov/trace.lcov_base -a ../afl_out/cov/lcov/trace.lcov_info --output-file /tmp/tmppLxYHX
Traceback (most recent call last):
 File "/data/vagrant_vms/afl-cov//afl-cov", line 1190, in <module>
  sys.exit(main())
File "/data/vagrant_vms/afl-cov//afl-cov", line 91, in main
  return not process_afl_test_cases(cargs)
File "/data/vagrant_vms/afl-cov//afl-cov", line 208, in process_afl_test_cases
cov, cargs)
File "/data/vagrant_vms/afl-cov//afl-cov", line 314, in coverage_diff
  new_cov = extract_coverage(cov_paths['lcov_info_final'], cargs)
File "/data/vagrant_vms/afl-cov//afl-cov", line 462, in extract_coverage
  with open(lcov_file, 'r') as f:
 IOError: [Errno 2] No such file or directory: '../afl_out/cov/lcov/trace.lcov_info_final'`

Could anyone shed some lights on what/how does this error happen? Thank you!

Hi,
I'm trying to run AFL-cov on a finished AFL test output.
I ran AFL using the command /path/to/afl-fuzz -i inputDir/ -o outputDir/ /path/to/program @@ , and I compiled the program using AFL's clang compiler.

When using AFL-cov, I'm trying to follow the examples in the README, so after compiling the program separately with coverage support, I tried to run AFL-cov with these commands:

$ cd /emptyDir
$ /path/to/afl-cov -d /path/to/outputDir --coverage-cmd "/path/to/program -f AFL_FILE" -c /path/to/recompiled/program --enable-branch-coverage

I get this error:
[*] Could not find an executable binary --coverage-cmd ' ~/Applications/curl/AFLbuild/src/curl -f AFL_FILE'

The path I give is exactly the same as the path I used to start AFL earlier. How can I fix this? If I just specify the directory where the program is in and not the direct path to the program, I get the same error.

Any help is appreciated. Thank you.

Currently, afl-cov needs the output file of afl-fuzz to calculate coverage, which means that two processes are required. But if there are multiple files and the target program is very large, the calculation speed will be very slow. Does afl-cov support reading the bitmap to calculate the coverage, which will speed up the calculation?

Or, to think about this more simply, afl-cov now generates the coverage of each file. Which option allows me to get the overall coverage without so many details?

When I follow the README method and first run afl-cov with the --live parameter and then run afl-fuzz, I found that the binary compiled with the "-ftest-coverage -o afl_test_cov" option failed to run. But binary files compiled directly with the "-g" option can run successfully. The following is the error：

~/afl-cov$ afl-fuzz -T ./afl_test_cov -t 1000 -i ./in -o ./sync_dir ./afl_test_cov -a -b -c
afl-fuzz 2.51b by <[email protected]>
[+] You have 4 CPU cores and 4 runnable tasks (utilization: 100%).
[*] Checking CPU core loadout...
[+] Found a free CPU core, binding to #0.
[*] Checking core_pattern...
[*] Setting up output directories...
[+] Output directory exists but deemed OK to reuse.
[*] Deleting old session data...
[+] Output dir cleanup successful.
[*] Scanning './in'...
[+] No auto-generated dictionary tokens to reuse.
[*] Creating hard links for all input files...
[*] Validating target binary...

[-] Looks like the target binary is not instrumented! The fuzzer depends on
    compile-time instrumentation to isolate interesting test cases while
    mutating the input data. For more information, and for tips on how to
    instrument binaries, please see /usr/local/share/doc/afl/README.

    When source code is not available, you may be able to leverage QEMU
    mode support. Consult the README for tips on how to enable this.
    (It is also possible to use afl-fuzz as a traditional, "dumb" fuzzer.
    For that, you can use the -n option - but expect much worse results.)

[-] PROGRAM ABORT : No instrumentation detected
         Location : check_binary(), afl-fuzz.c:7148

while trying to use afl-cov to test a dlopen()ed library, I think i discovered a small bug in the check for properly providing instrumented executables.

namely the break in L894 should be indented to the inner if.

reasoning

the coverage-cmd might contain multiple executables, with only one of them being instrumented.
the current test will stop after checking the first word that is also an executable, and if this one is not instrumented, found_code_cov_binary will stay False, thus leading to early abort.

i'm not providing a PR, given that the fix is trivial and i'm not sure whether i'm actually right.

I tried to use afl-cov to statistic the coverage while execution python-afl-fuzz, but it cannot work well.
Could you provide the using guide for python-afl-fuzz?

Hi there,
Firstly, thank you for your great work!
I’m thinking if you have any interest in extending your work to analyze the variable behaviors of fuzzing targets?
If we can identify which part of source code is responsible for indeterminancy and choose to not instrument that part, we can greatly improve the stability of fuzzing.
If you have any interest, I would like to help.
Cheers,

is this a typo to use is_dir to check file exists?

def get_cycle_num(id_num, cargs):

    ### default cycle
    cycle_num = 0

    if not is_dir(cargs.afl_fuzzing_dir + '/plot_data'):
        return cycle_num

the function is_dir is defined to check if the argument is a directory and not a file:

def is_dir(dpath):
    return os.path.exists(dpath) and os.path.isdir(dpath)

Should probably add a command line argument (maybe enabled by default) to not keep all of the raw lcov results files.

Links to final reports break when -d is specified with a relative path instead of an absolute path.

code

for part in cargs.coverage_cmd.split(' '):
            if part[0] == ' ' or part[0] == '-':
                continue
            if (which(part)):
                found_exec = True
                if not cargs.disable_gcov_check and is_bin_gcov_enabled(part, cargs):
                    found_code_cov_binary = True
                    break

if coverage_cmd has multi space, the value 'part' will get a '' and making binary exit.like this

afl-cov -d /out --live --coverage-cmd "LD_LIBRARY_PATH=../lib/  ./tiffinfo -D AFL_FILE"   --code-dir /src/libtiff --genhtml-per-min 1440 --background
Traceback (most recent call last):
  File "/usr/local/bin/afl-cov", line 1236, in <module>
    sys.exit(main())
  File "/usr/local/bin/afl-cov", line 76, in main
    if not validate_cargs(cargs):
  File "/usr/local/bin/afl-cov", line 962, in validate_cargs
    if not is_gcov_enabled(cargs):
  File "/usr/local/bin/afl-cov", line 931, in is_gcov_enabled
    if part[0] == ' ' or part[0] == '-':
IndexError: string index out of range

Python complains that the file "lcov-web-final.html" already exists and kills the whole program, as soon as genhtml finishes the second report (around line 600).

http://stackoverflow.com/a/27103129 has a nice short function to fix this issue that works beautifully as a drop-in replacement.

This will result in errors from lcov, which are shown as 'command returned status 255'. Just a heads-up for everyone trying to run this with a newer version of gcc.

Add a --last-cmd option to re-run afl-cov with the same command line args as the previous execution.

why genhtml is not generating on lcov 1.14 whts the reason and how to fix asap?

From your README, to run the program that need AFL_FILE as stdin:

$ cd /path/to/project-gcov/
$ afl-cov -d /path/to/afl-fuzz-output/ --live --coverage-cmd \
"cat AFL_FILE | LD_LIBRARY_PATH=./lib/.libs ./bin/.libs/somebin -a -b -c" \
--code-dir .

but this code will treat the first word cat as the exec part, leading to error message:

    Non-zero exit status '1' for CMD: /usr/bin/readelf -a cat

I would recommend change the README to:

afl-cov -d /path/to/afl-fuzz-output/ --live --coverage-cmd \
"LD_LIBRARY_PATH=./lib/.libs ./bin/.libs/somebin -a -b -c < AFL_FILE " \
--code-dir .

or change the code to add a parameter for specifing the exec part.

The plot_data file can be used by afl-cov to show what new code has been covered on a per-cycle basis.

See https://docs.python.org/2/library/tempfile.html for reference - currently afl-cov seems to write a lot of temporary data to my SSD, at least some of it could be prevented (and it might even speed up the process!) by writing to tempfiles instead.

Should check for availability of required commands like lcov and genhtml.

please tell me how to look for the cause or problem
I start without a flag (--live) or with it there is no difference all the same hangs approximately at processing 1000\25000
tell me other programs to build coating in addition to the afl-com
thanks!

logr("[-] Coverage file '%s' does not exist, skipping." % lcov_file)

logr() takes exactly 3 arguments (1 given).

afl-cov

check 314 and 458 lines of the code

my afl command is this:
afl-fuzz -t 300+ -m none -i solidity_input_part -o output -Q -- solc --bin -o --overwrite @@
my afl-cov command is this:
./afl-cov -d output --coverage-cmd "/usr/bin/solc AFL_FILE" -c . --enable-branch-coverage --overwrit
but I get an error:
[*] Could not find an executable binary with code coverage support ('-fprofile-arcs -ftest-coverage') in --coverage-cmd '/usr/bin/solc AFL_FILE'

afl-fuzz argv written to fuzzer_stats might provide a nice way to validate the afl-cov --coverage-cmd, although env variables would likely be an issue.

Hi,

Any idea on how to run afl-cov when fuzzing CGC DARPA binaries? Thanks.

Use the fuzzer_stats file to see if afl-fuzz is currently running instead of the current strategy of grep'ing the process table.

I have using fuzzed mysql and generated some seed file.Now I want to use afl-cov to collect mysql code coverage.I use the following command:
/home/mysql/afl-cov/afl-cov -d /home/mysql/fuzzing/fuzz_output --enable-branch-coverage -c . -e "cat AFL_FILE |./bld_aflcov/bin/mysqld" or /home/mysql/afl-cov/afl-cov -d /home/mysql/fuzzing/fuzz_output --enable-branch-coverage -c . -e "cat AFL_FILE |./bld_aflcov/bin/mysql"
But it doesn't work

In my case, afl-fuzz is much faster in generating new test cases than lcov can generate output. A mode where counters are zeroed, then afl-cov runs all queue items it knows about and then runs the rest of the lcov commands to generate a single report would be a great thing to have.

afl-cov line 438 combined "base" and "info" into "final". So "final" contains data about zero coverage. However, the next step, afl-cov line 444, generate "final" again using "info" as input. Therefore, if there are some files with zero coverage, that file will be missing in the final html output.

If you're using "-M" and "-S" options to afl-fuzz, you get output directories "outputs/name1/" "outputs/name2/" etc. with queue directories under these. This confuses afl-cov which expects "outputs/queue" to exist.

I am using afl-cov on Mac OS. It seems that afl-cov depends on readelf, which does not exist on Mac. What would you suggest to deal with this?

$ afl-cov -d ./out --coverage-cmd "cat AFL_FILE | ./a.out" -c .
[*] Need a valid path to readelf, use --readelf-path

Look for the --rc command line argument to the lcov command to enable branch coverage.

Hi,

I just noticed that during live mode no HTML output is generated. It would be nice if the web output was also generated when monitoring a running fuzz job and furthermore if it was updated whenever the coverage status is printed to the console. Doing this less frequently would be ok too. I think updating the HTML every few minutes or so would be fine.

This could eliminate the overhead for regenerating the complete HTML output for every update coverage-wise. Additionally the user could simply hit F5 in the browser to check the updated coverage reports.

Thanks!

A comprehensive test suite should be added.

I followed the instructions given in the workflow section. Running afl-cov gives me following error:

$ afl-cov -d /home/numair/Documents/faultlocalization/evaluation/RealWorld/AFL-FUZZING/findings --live --coverage-cmd "cat AFL_FILE | LD_LIBRARY_PATH=./lib/.libs ./bin/.libs/somebin -a -b -c" --code-dir .
[-] Sleep for 60 seconds for AFL fuzzing directory to be created...
    
*** Imported 2 new test cases from: /home/numair/Documents/faultlocalization/evaluation/RealWorld/AFL-FUZZING/findings/queue

    [+] AFL test case: id:000000,orig:test1.txt (0 / 2), cycle: 0
lcov: ERROR: no valid records found in tracefile /home/numair/Documents/faultlocalization/evaluation/RealWorld/AFL-FUZZING/findings/cov/lcov/id:000000,orig:test1.txt.lcov_info
Traceback (most recent call last):
  File "/usr/bin/afl-cov", line 724, in <module>
    sys.exit(main())
  File "/usr/bin/afl-cov", line 72, in main
    return not process_afl_test_cases(gpaths, cargs)
  File "/usr/bin/afl-cov", line 140, in process_afl_test_cases
    gen_coverage(gpaths, cov_paths, f, cargs)
  File "/usr/bin/afl-cov", line 447, in gen_coverage
    + cov_paths['lcov_info_final'], cargs, WANT_OUTPUT)
  File "/usr/bin/afl-cov", line 536, in run_cmd
    out = subprocess.check_output(cmd.split())
  File "/usr/lib/python2.7/dist-packages/subprocess32.py", line 635, in check_output
    raise CalledProcessError(retcode, process.args, output=output)
subprocess32.CalledProcessError: Command '['lcov', '--rc', 'lcov_branch_coverage=1', '--no-checksum', '-r', '/home/numair/Documents/faultlocalization/evaluation/RealWorld/AFL-FUZZING/findings/cov/lcov/id:000000,orig:test1.txt.lcov_info', '/usr/include/*', '--output-file', '/home/numair/Documents/faultlocalization/evaluation/RealWorld/AFL-FUZZING/findings/cov/lcov/id:000000,orig:test1.txt.lcov_info_final']' returned non-zero exit status 255

What am I doing wrong ?

Hi, I've encountered an issue with code coverage in C++ code that uses coroutines. Here is a repository with a demo on how to reproduce the described situation:

https://github.com/PiotrConclusive/afl-cov-coroutines/tree/master

I've been using afl-g++ in a Docker environment provided by the afl project:
https://github.com/AFLplusplus/AFLplusplus/blob/stable/Dockerfile

In the repository, there is an attached report at https://github.com/PiotrConclusive/afl-cov-coroutines/blob/master/aflOut/cov/web/index.html, where you can see that some lines in coroutine functions appear white.

Emulate the afl-fuzz fuzzer_stats file with an afl-cov version that writes at least the pid and argv to cov/cov_stats

Can afl-cov be used in conjunction with afl and unicorn to perform coverage statistics on closed source binaries?

I've been trying to use afl-cov while running multiple instances of afl in parallel but it doesn't seem to sense that afl started

Hi,I run afl-cov on ubuntu 14.04,but it reported that lcov_info_final does't exist..
AFL test case: id:000001,src:000000,op:flip1,pos:0,+cov (1 / 273), cycle: 0
Traceback (most recent call last):
File "/usr/local/share/afl-cov/afl-cov", line 1066, in
sys.exit(main())
File "/usr/local/share/afl-cov/afl-cov", line 77, in main
return not process_afl_test_cases(cargs)
File "/usr/local/share/afl-cov/afl-cov", line 174, in process_afl_test_cases
f, cov, cargs)
File "/usr/local/share/afl-cov/afl-cov", line 261, in coverage_diff
new_cov = extract_coverage(cp['lcov_info_final'], cargs)
File "/usr/local/share/afl-cov/afl-cov", line 416, in extract_coverage
with open(lcov_file, 'r') as f:
IOError: [Errno 2] No such file or directory: '/root/workspace/afl_binutils/binutils-2.26/afl_out/cov/lcov/afl_out/id:000001,src:000000,op:flip1,pos:0,+cov.lcov_info_final'

Afl-cov command was:

afl-cov -d .../fuzzout/ --coverage-cmd "./binary 1 AFL_FILE" --code-dir ~/project-gcov/

(started in ~/project-gcov/binary)

Note the missing index.html linked by lcov-web-final.html below:

fuzzout/cov/web$ ls -l
lcov-web-final.html -> /home/.../fuzzout//cov/web/f8/id:000373,sync:f2,src:000383/index.html

fuzzout/cov/web$ cd /home/.../fuzzout//cov/web/f8/id:000373,sync:f2,src:000383

cov/web/f8/id:000373,sync:f2,src:000383# ls
amber.png  emerald.png  gcov.css  glass.png  root  ruby.png  snow.png  updown.png

The only html file produced is a mock-util.c.gcov.html that only has coverage info for that mock-util.c file. Is this due to some failure to create the files or am I using some unsupported way of running afl-cov ?

Hi Michael,

for the Debian package, we've patched afl-cov to import subprocess32 [1] instead of Python 2 stdlib subprocess, which is highly recommended to replace on POSIX systems. Do you would like to apply that upstream?

[1] https://pypi.python.org/pypi/subprocess32/

i'm currently trying to use afl-cov to test a dlopen()ed library.

unfortunately the safety-nets in afl-cov give me a hard time, esp. the check whether the cmdline contains instrumented executables.

my main executable is not instrumented, but it loads a configuration that will trigger the dlopen()ing (and use) of an instrumented binary - so the coverage-cmd doesn't contain any hint about the binary in question.

rather than patching the afl-cov script, would it be possible to add a cmdline-flag to disable the found_code_cov_binary-test?