uds-se / fuzzingbook Goto Github PK

Is your feature request related to a problem? Please describe.
I'd like to quickly navigate between chapters

Describe the solution you'd like
A pull down menu on top would be great

Click on some button to get the solution to an exercise

Describe the bug
The jupyter notebook In binder misses the solution2 and toc2 extensions.

The link in the menu points to non-beta notebook; but there are beta notebooks in beta.

While using AFLSmartSchedule parser can not be found in parsable(self, seed):

class AFLSmartSchedule(PowerSchedule):
...
    def parsable(self, seed):
        """Returns the substring that is parsable"""
        table = self.parser.chart_parse(seed.data, parser.start_symbol())
        cols = [col for col in table if col.states]
        return seed.data[:len(cols)-1]
...

Suggested Fix:
table = self.parser.chart_parse(seed.data, self.parser.start_symbol())

Have a "docs/beta" site (possibly redirected to from beta.fuzzingbook.org) that contains all docs, even the ones that are not ready yet.

In the dev tree, create a "beta" folder as alternative to "docs" and add your pages there?

After the XML parser comparison in GreyboxFuzzer chapter below the graphics is written:
"Both greybox fuzzers clearly outperform the greybox fuzzer".

Fix: "Both greybox fuzzers clearly outperform the blackbox fuzzer"

Hi,
I installed the python module both via pip manually
pip install fuzzingbook
and through the PyCharm GUI.

When I want to import something, e.g. the sample import
from fuzzingbook.Fuzzer import RandomFuzzer
I always receive
ModuleNotFoundError: No module named 'fuzzingbook'

I am running python 3.6

The current nav bar scrolls away with the page; this is a problem when navigating to sections

We need a way to open the appropriate notebook right in jupyter lab. The example in https://github.com/binder-examples/jupyterlab does not seem to work.

In the function

def parsable(self, seed):
    """Returns the substring that is parsable"""
    table = self.parser.chart_parse(seed.data, parser.start_symbol()) 
    cols = [col for col in table if col.states]
    return seed.data[:len(cols)-1]

the global variable 'parser' is used instead of a class variable with the same name.

In the book, this is not noticeable as 'parser' and 'self.parser' are initialized with the same object, but on import, this function returns the error that 'parser' is uninitialized.

Grammar/Typos

1. how one identify which

2. Given a function gcd and two test cases evaluate,

3. However, that that introduces

4. assume that each token in the program have

5. What if actually try to

6. test cases that can detect a single change in isolation is

7. output, Lines

8. discoverd

This paragraph appears twice on the same page:

The Mutator provides the base class for...

While the book is not done, we will have plenty of links to (yet) non-existent pages. Have a 404 page that explains this:

https://help.github.com/articles/creating-a-custom-404-page-for-your-github-pages-site/

Describe the bug
HTML output of Python commands always contains one extra line (before )

To Reproduce
Any output cell

Is your feature request related to a problem? Please describe.
Make it easy to appropriately cite the book and individual chapters

Describe the solution you'd like
A "cite" link at the bottom would issue a BibTeX entry to cite the chapter

Syntax diagrams (also called railroad diagrams) are a way to visually represent a context-free grammar. It would be great if we can have a display_grammar analogue for display_tree that can display the syntax-diagram corresponding to a grammar.

A list Python libraries that can produce syntax-diagrams in SVG:

• syntrax
• syntax-diagram

Is your feature request related to a problem? Please describe.
Python beginners may have trouble understanding what's going on in a Python program. If we could add a means to automatically show the meaning of a python keyword, that would be great!

Describe the solution you'd like
Hovering or clicking on a code word that is also in the Python documentation index could link to that word.

Typos/Grammar

1. In Python, we can can squeeze long

2. Lambda allows for quick definitions unnamed functions.

3. Both greybox fuzzers clearly outperform the greybox fuzzer.

"Wrong" conclusion

As expected, the boosted greybox fuzzer (with the exponential power schedule) achieves coverage much faster.

The graph presented in the current version indicates the opposite. (Further down a tldr explains the variance in experiments but imho the graph should match the description.)

Github traffic stats only cover the "proper" Github parts, not the pages hosted via Github pages (such as all of fuzzingbook.org). Include Google analytics such that we can get an idea on our reader demographics, track which chapters are most popular, and inflate our egos proportionally to traffic.

The first expansion strategy used by the GrammarFuzzer tries to widen the derivation tree until reaching min_nonterminals unexpanded non-terminals.

There are cases however where it is not possible to create a tree with enough unexpended non-terminals, potentially leading to infinite loops if recursive production rules are chosen, e.g.:

from fuzzingbook.GrammarFuzzer import GrammarFuzzer
from fuzzingbook.ExpectError import ExpectTimeout

grammar = {
    '<start>': ['<A>'],
    '<A>': ['a<A>', 'a'],
}

fuzzer = GrammarFuzzer(grammar, min_nonterminals=2, max_nonterminals=10, log=True)

with ExpectTimeout(2):
    fuzzer.fuzz()

Actually, the current implementation maximizes the risk to encounter these infinite loops because recursive non-terminal expansions are assigned an infinite cost (so they will always be picked first by the max-cost expansion algorithm).

This entire expansion step feels weird to me. Why not go for a (best effort) minimum width for the derivation tree (terminals included) instead of saying that we want to only consider the non-terminals?

... that automatically fixes all code cells in a notebook.

Describe the bug
In

• https://www.fuzzingbook.org/html/Fuzzer.html#Missing-Error-Checks

• fuzzingbook/notebooks/Fuzzer.ipynb

  Line 1061 in 6db8144

  "char read_next_nonspace() {\n",

• fuzzingbook/docs/notebooks/Fuzzer.ipynb

  Line 1196 in d3845db

  "char read_next_nonspace() {\n",

There is a function char read_next_nonspace(). The name of the function implies that it will return the next non-whitespace character, and the description of the function in the paragraph of text that precedes the code fragment says so too;

Now assume a programmer is scanning the input for the next character, skipping space characters

The function itself is implemented as:

char read_next_nonspace() {
    char lastc;

    do {
        lastc = getchar();
    } while (lastc != ' ');

    return (lastc);
}

The next paragraph of text goes on to say:

What happens if the input ends prematurely, as would perfectly be feasible with fuzzing? Well, getchar() returns EOF, and keeps on returning EOF when called again; so the code above simply enters an infinite loop.

In isolation, this latter paragraph is stating the truth; this function does indeed get stuck in an infinite loop whenever getchar() returns EOF to it.

However, if we look at what happens other than that, we see that read_next_nonspace() is actually returning the first space, if any, that it finds, and skipping non-space characters. This is opposite to what the name of the function and its description implies that the function should do.

The fix for that issue is to change the loop condition from lastc != ' ' to lastc == ' '. Giving us:

char read_next_nonspace() {
    char lastc;

    do {
        lastc = getchar();
    } while (lastc == ' ');

    return (lastc);
}

And with that we no longer get stuck on EOF either.

Furthermore it is my opinion that the bug-fixed implementation above is what the complete implementation of that function should look like, and that no handling of EOF should be done inside this function. Instead, EOF should be handled by code that makes use of read_next_nonspace().

A complete program that makes use of the bug-fixed function can be implemented as

#include <stdio.h>

char read_next_nonspace ()
{
  char lastc;

  do
  {
    lastc = getchar();
  } while (lastc == ' ');

  return (lastc);
}

int main ()
{
  char next_nonspace;

  while ((next_nonspace = read_next_nonspace()) != EOF)
  {
    putchar(next_nonspace);
  }
}

As such I would argue that the originally given code fragment was never a case of a missing error check to begin with, because the EOF handling does not belong in that function. We can view read_next_nonspace() as a sort of "stream filter" with its "stream source" connected within it that reads from stdin and filters out spaces. EOF is a sentinel value (and usually implemented as a macro that resolves to -1) of great importance when dealing with filters but its presence should not affect this filter itself.

We can see this even more clearly if we separate the the filter from its source.

Flipping things around a little bit we get something akin to this: https://github.com/ctsrc/sources-and-filters/blob/2936e6501ec6d1cadecfd61601728d9cd6c01e16/filter-space.c

With that we can then go on to chaining multiple filters together etc. And hopefully the compiler writer people will have written their compilers to optimize away the overhead of our nested function calls and pointer derefs ;)

Anyway, I digress. The point of the bug report was to say that I believe that the originally given example was not a case of missing error handling, and that a different code fragment needs to be written that will properly illustrate the issue.

Change all references to fuzzingbook. "gstbook/" could become "fuzzingbook_utils".

In particular, turn off cell menu

Describe the bug
In 'WhenToStopFuzzing' chapter there is a typo in the code. Basically when you build the k_book the last trigram is not taken in account

To Reproduce
Steps to reproduce the behavior:

1. Go to WhenToStopFuzzing notebook in the part where the k_book is created and add at the end the lines:

print(trigrams[-2])
print(k_book[trigrams[-2]])
print(trigrams[-1])
print(k_book[trigrams[-1]]) 

2. Execute the code
3. See error

Expected behavior
The k_book is supposed to contain the trigram

Screenshots

Additional context
The for loop should be corrected from
for i in range(1, len(trigrams)):
to
`for i in range(1, len(trigrams)+1):'

I know that's only a typo, but I hope I helped more than bother you.

EDIT:
I noticed that if you act like that the normalization factor is ruined. The normalization factor should be corrected in
'log(1 + 1 / i) / log(263+1)'
as the last i value is 263+1 and not 26**3

downloaded (all)
could not find PICS/Sitemap.svg

Otherwise (super) fine notebooks

Greets
PKHG

Describe the bug
PrintRunner returns UNRESOLVED although the Fuzzer chapter states:

A PrintRunner will simply print out the given input and return a PASS outcome

To Reproduce
Run the example from the Fuzzer chapter:

>>> random_fuzzer = RandomFuzzer(min_length=10, max_length=20, char_start=65, char_range=26)
>>> print_runner = PrintRunner()
>>> random_fuzzer.run(print_runner)
ALZBCUTZSBSXWUFTBIO
('ALZBCUTZSBSXWUFTBIO', 'UNRESOLVED')

See:

Is your feature request related to a problem? Please describe.
When importing modules within Python (not Notebooks), all notebook statements get executed. This leads to spurious output (all the prints from the notebook) and lots of burnt CPU cycles.

Describe the solution you'd like
Prefix cells with non-exportable statements (e.g. print statements, expressions) with
if name == “main”:
The rules for that could be the same as with notebook imports (see fuzzingbook_utils)

See https://medium.com/@ageitgey/learn-how-to-use-static-type-checking-in-python-3-6-in-10-minutes-12c86d72677b

• Advantages: Make code more readable, in particular by extending function signatures. Allow for static type checking.
• Disadvantages: Requires Python 3.6; harder to back port code to Python 2. (Could use Python 2 Style comments though)

Describe the bug
$\sqrt{x}$ is not rendered in HTML notebooks.

To Reproduce
Have a cell with $\sqrt{x}$. The \sqrt is simply ignored.

Expected behavior
I should see $\sqrt{x}$.

Describe the bug
The export to PDF fails for the following notebook: WhenToStopFuzzing

To Reproduce
Steps to reproduce the behavior:

1. Go to WhenToStopFuzzing notebook
2. Export to PDF
3. Cry because you can't export the notebook

Expected behavior
Export should render and download a PDF file

Screenshots
Won't add. I'm sending a fix in a second

Desktop (please complete the following information):
Irrelevant, as far as I know

Additional context
No need

... and is improperly linked as [Pezz{'e} et al, 2008]

We may need this if my binder.org is overloaded

Describe the bug
When exporting PDF, any HTML output is not included

To Reproduce
Produce a PDF from WebFuzzer.ipynb

Expected behavior
The HTML interaction should be rendered in print

This could be fixed by running a headless Firefox (see GUIFuzzer.ipynb) and have it render the HTML parts.

Using inspect.getsource() to retrieve the source code of functions does not work if

• the function is defined in a notebook, and
• we are calling inspect.getsource() from a notebook.

The reason is that inspect.getsource() looks for .py files, not .ipynb. We patch this for import of modules, but not for inspect.

Solution: Provide an alternative to inspect.getsource() (say, fuzzingbook_utils.getsource()) which looks for defs in .ipynb files (or creates a temporary .py file that getsource() could use). See the inspect source code for details.

Describe the bug
Because the GrammarCoverageFuzzer inherits from the GrammarFuzzer, it still uses the expansion strategy based on costs. This can result in the fuzzer avoiding completely parts of the grammar because it first selects expansions with maximum cost. For a fuzzer that pretends to cover the grammar that is pretty confusing.

To Reproduce
For example, the following code will never produce an 'a':

from fuzzingbook.GrammarCoverageFuzzer import GrammarCoverageFuzzer

grammar = {
    '<start>': ['<A>', '<B>'],
    '<A>': ['a'],
    '<B>': ['b<C>', '<D>'],
    '<C>': ['c'],
    '<D>': ['d']
}

fuzzer = GrammarCoverageFuzzer(grammar, min_nonterminals=5, max_nonterminals=10)

for i in range(100):
    print(fuzzer.fuzz())
    print('-----------')

Allow the pages to be read in dark mode. See also

https://web.dev/prefers-color-scheme

... but should come in multiple colors. The touch icon works, though.

Provide for local Dockerfile creation (using "make docker"?).

Have a nicer design on mobile sites (larger fonts, responsive menu, etc.)

For probabilistic and constraint-based grammars, syntax diagrams should show the annotations, too.

Check full notebook path

Describe the bug
There is a dead hyperlink.

To Reproduce
In readme,

"See the table of contents for those chapters that are already done."

A hyperlink in the sentence, https://github.com/uds-se/fuzzingbook/blob/master/00_Table_of_Contents.ipynb, is dead.

Regards,
Dongsun.

Make sure bc is installed, too

www.fuzzingbook.org works, but fuzzingbook.org does not

Have a more modern web design; possibly with (1) responsive design and (2) navigation

Typos/Grammar

1. We can use this distance value as out fitness function

2. neighbor vs. neighbour

3. there is a gradient that takes is straight to an optimal solution

4. the false distance of is 0 by definition

5. it is possible that the same test executions a condition multiple times

6. corresponding to the two conditions, and the and with which they are

7. produce this instrumented varion

8. Python provides API to

9. for which first retrieve the

10. The better the fitness value an individual

11. small number of randomly chosen individual

Funny that the algorithm finds a unicode sequence that changes the output order for "%s: %.4f":

'\ua7e9\ue9e0\ue7de턤댪囿厠‐\ue0c5ﴌ': 2.9999

Describe the bug
On mobile (touch) devices such as the iPhone, one can choose from the top menu, and while choosing, scroll down. The top menu then disappears (as should be), but the submenu stays open, which one can see if one scrolls up again.

To Reproduce
See above.

Expected behavior
The previously open submenu should be closed as it scrolls away.

Right now, the Docker image in mybinder does not include the Python graphviz module. (see, for instance, the Grammars notebook)