MD to PDF about docker-security HOT 49 CLOSED

Hi @drwetter,
You can check how we did it for Go Secure Coding Practices: we're using gitbook-cli v2.3.0.
In the dist folder you'll find the resulting output formats: ePub, MOBI and PDF.

GitBook might also be an option: it is free for open-source & non-profit teams.

Cheers,
Paulo A. Silva

from docker-security.

from docker-security.

Oh, yes thanks. Sphinx rings a bell too.

from docker-security.

Thanks, @PauloASilva

we're using gitbook-cli v2.3.0.

Was that done automatically or did it require manual work?

from docker-security.

Hi @drwetter,
No manual work required: you just have to run

$ npm install && npm run build

We can do it locally or automate it, for instance, on Travis CI using GitHub Hooks.

Cheers,
Paulo A. Silva

from docker-security.

That sounds good!

Mind to help?

from docker-security.

That sounds good!

Mind to help?

Not at all, I will be glad to do it 😄

from docker-security.

Sounds great.

How about starting with it early? I created a separate branch (gitbook) for it so that we can play with it before it looks like it's "the" document?

from docker-security.

I'm on it already: I forked your repo and soon I will do a pull request.

I did an initial test but the build failed due to some parsing errors.
As soon as I get some output, I'll share it with you so that we can decide how to proceed.

Cheers,
Paulo A. Silva

from docker-security.

from docker-security.

Hi @drwetter,
We have a first draft: please check it on my fork.
The resulting formats (PDF, ePub, MOBI) are in the dist root folder.

If we decide to move forward with gitbook, we should:

• Create a Makefile to build the book inside an ephemeral docker container. Now I did it by hand

  [pauloasilva@XXX Docker-Security]$ docker run -it --rm -v $(pwd):/app node:current-slim bash -c "cd /app && apt-get update && apt-get install -y calibre && npm i && npm run build"
  

• Rename the 000 - Introduction chapter (gitbook expects README.md to be the Introduction chapter)
• Validate D04 - Secure Defaults and Hardening.md Markdown as there are some parsing problem regarding dots on a command line sample marked with FIXME.

Please share your thoughts on this.

Cheers,
Paulo A. Silva

from docker-security.

from docker-security.

Hi @drwetter,
Happy Easter to you too!

Cheers,
Paulo A. Silva

from docker-security.

Hi Paulo,

thanks!

• Rename the 000 - Introduction chapter (gitbook expects README.md to be the Introduction chapter)

can't this be configured?

Speaking of it: the font (calibre) looks in my eyes rather painful. A sans-serif font would do much better to the eye. :-)

Validate D04 - Secure Defaults and Hardening.md Markdown as there are some parsing problem regarding dots on a command line sample marked with FIXME

I was working on that last week and correct that today or later. If it holds you up and you don't want to hotfix it in your fork first, please let me know.

Upon opening the PDF I saw the graphics and said "wow" :-)

Cheers, Dirk

from docker-security.

D04 should be fine for you now.

from docker-security.

Hi Dirk,

• gitbook-cli supports changing the default introduction README.md, but the file name should match the following RegEx ^[a-zA-Z-._\d,\s]+$ what 000 - Introduction.md doesn't.
  To be honest I don't like the current naming convention because:

  • it includes spaces (and symbols), requiring escaping on CLIs
  • it mixes lower and upper case letters, what may cause troubles on case sensitive file systems

  Would you agree renaming files as follows?

  • 000 - Introduction.md -> 000-introduction.md
  • D03 - Network Separation and Firewalling.md -> d03-network-separation-firewalling.md
  • E11 - What'\''s Next?.md -> e11-whats-next.md

• Now I am forcing the fontFamily to sans-serif (gitbook-cli PDF options)

• The cover was an Easter Egg 😄

I'll pull your changes and see if they solve the markdown parsing issue.

Best,
Paulo A. Silva

from docker-security.

Hi Paulo,

the Easter Egg arrived on time ;-)

I can work on Readme and the Introduction to rename them.

As far as the naming of the files in general are concerned: I guess it's a matter of taste. At least ^[a-zA-Z-._\d,\s]+$ matches 000 - Introduction.md, see e.g. https://regex101.com/. I am assuming this pattern applies for all filenames?

I am a friend of readable filenames. Why should a human sacrifice readability to a machine? Just because some sloppy shell programmers forgot to quote properly?

Don't know what OS you're using? Even on Linux (which I use) I only have little problems with such filenames. On MacOS and Windows they are common since a longer while back.

In E11 the question mark and quote was too much maybe. I pushed a change.

Cheers! Dirk

from docker-security.

Hi Dirk,
Please don't change anything on your side because I am not sure what's wrong: although gitbook complains about the file name I am not being able to use a value other than README.md.

I'll let you know when I have figured out what's wrong and how to proceed with the PoC.

I am a Linux user too.
After a few years of software development, working with other developers with different OS preferences, I had to adopt a standard for file naming to avoid undesired problems.

The most recent incident happened during the OWASP Top 10 translation.
I got a volunteer who is a MacOS user. In the repository there are two files sharing the same name: one lower case, the other upper case. The volunteer gave up when he started receiving errors from git.

I am aware of MacOS/git settings regarding file system case sensitivity but I don't think it is fair to ask someone to change his system to contribute. Nowadays I prefer to sacrifice readability a little to avoid facing such boring and counter-productive problems.

Cheers,
Paulo A. Silva

from docker-security.

Hi Dirk,
In fact, there's a GitBook bug performing file name validation 😞
I will fix it and do the PR.

Meanwhile, please check the resulting PDF (and other formats): no need for file name renaming and the font family issue was also addressed.

Cheers,
Paulo A. Silva

from docker-security.

PDF checked and looks great so far! Thanks!

More later...

from docker-security.

@PauloASilva : How about form feeds?

Is there anything what I can/need to do? On some pages the headline looks more like a footnote, see . e.g. D01, p7:

How can I find out?

Configuration

Then comes the form feed.

from docker-security.

You're asking for control over the page breaks, right?

I did a quick search and according gitbook PR#183 it should be possible, but I don't see these options in the current master branch.

I need to further investigate.

from docker-security.

yes, form feed is maybe more the technical term / nerd speak for a character which does a page break.

from docker-security.

Hi @drwetter,
GitBook does not offer a way to manually control form feeds. The PDF options mentioned before are just to control form feeds style: whether it should be a page break, a rule or both.

Of course we can work around it using empty paragraphs but then we're defeating the premise of focusing solely on the content, leaving the styling thing to the tool.

Do you think this is a major issue?
Footnotes have their own style: please check D03 p.13.

Cheers,
Paulo A. Silva

from docker-security.

Hi @PauloASilva ,

unfortunately it would be an issue.

You're sure it hasn't been integrated? The PR you were referring to was 2015. If I look at the code, I cannot find the files of that PR anymore but if I search the master (https://github.com/GitbookIO/gitbook/search?q=pageBreaksBefore&unscoped_q=pageBreaksBefore) I see those changes.

Maybe somebody just forgot to document that change?

Cheers, Dirk

from docker-security.

Hi @drwetter,
Maybe my explanation was not clear. In fact those options exist on master branch but they do a different thing: instead of creating a page break when an h1 is found, we can change it to --- (rule).

When I first checked this I was not able to see how this could be used but now, while writing, maybe it could be an option: we can start each chapter with --- causing the page break as well for "special cases" (h2, h3, ...) like the one you spotted.

Let me give it another try.

Cheers,
Paulo A. Silva

from docker-security.

If I read that correctly it seems "pageBreaksBefore": "<someusualchar>" would do it.

from docker-security.

from docker-security.

Hi @drwetter
After reading Calibre documentation and play with PDF options
pageBreaksBefore does what we're looking for, with some limitations:

• === pagebreak === (//*[contains(text(), '=== pagebreak ===')])
  A page break is added before the marker, but the marker itself is also
  printed. Not an option.
• <pagebreak /> (//*[name()='pagebreak'])
  Does exactly what we want: we can now force the page break. Setting
  chapterMark to none won't prevent page breaks to be added before detected
  chapter (doesn't look like an issue)

Pleas check the latest PDF.

If we agree on moving forward with GitBook

1. I should fork and fix the regular expression issue
2. Build the book on a Docker container (I am already doing it locally but
   image needs to be shrunken so that it can be disposable)

Cheers,
Paulo A. Silva

from docker-security.

from docker-security.

Hi @PauloASilva ,

could you do a rebase pls? We didn't discuss pictures before, didn't we? Just merged the first picture.

Second request: Is manual work still required? I just looked @ your fork and it seems the PDF is not automatically build yet.

Cheers, Dirk

from docker-security.

from docker-security.

Hi Paolo,

no reason to be sorry -- almost everybody has to do payed work. ;) -- including myself.

First step for me would be just to have a look in your fork how it works with the picture and whether I need to do anything in the main repo. An automatic build of the document files would only makes sense if they look good, So maybe we can use your fork to get this ready first?

Cheers, Dirk

from docker-security.

from docker-security.

Hi @drwetter,
Please check my feature-build branch.
To test the build system and the resulting PDF, do:

$ git clone -b feature-build https://github.com/PauloASilva/Docker-Security.git
$ docker-compose run --rm builder

Please note that:

• book source files were moved into src sub-directory
• if you want to add additional page breaks, you can insert <pagebreak /> in the markdown files (e.g. D01 - Secure User Mapping)
• new "How to Build PDF version" section was added in the root README.md file
• when building for the first time a docker image will be create (build container is ephemeral)

Cheers,
Paulo A. Silva

from docker-security.

from docker-security.

from docker-security.

I downgraded the version in the yaml file yesterday.docker-compose run --rm build did it then. That's quite a container :-)

However it didn't go though for me, see below.

from docker-security.

Hi @drwetter,
The container is huge because of package calibre and its dependencies.

Please give it another try: the /build directory is now created during the image build with proper ownership and filesystem permissions.

Cheers,
Paulo A. Silva

from docker-security.

Hi @PauloASilva ,

thanks!

Still can't detect the picture.

book source files were moved into src sub-directory

Not sure I like that. Can't that be done just by the container, to copy?

Cheers, Dirk

from docker-security.

Hi @drwetter

Hi @PauloASilva ,

thanks!

Still can't detect the picture.

Maybe the picture path is wrong. I'll check it.

book source files were moved into src sub-directory

Not sure I like that. Can't that be done just by the container, to copy?

If you don't mind to mix building stuff (e.g. Dockerfile, docker-compose.yml) with the book source, we can put everything together, but then remind that sorting alphabetically (like GitHub WebUI does) will mix things together.

Please check the OWASP/API-Security repository structure.
The repository root is used for general stuff like the README.md, CONTRIBUTING.md, repo configurations (e.g. .gitignore, .gitattributes), "coding standards" (e.g. .editorconfig), etc.
Then we have:

• year: we plan to update de Top 10 every 2 or 3 years
• lang: we plan to track translations in this same repo
  • src: the source files (e.g. MD, PNG, ...)
  • dist: the distributable formats (e.g. PDF, ODT, ...)

Let me know whether you want to move the source files back to the repository root.

Cheers,
Paulo A. Silva

Cheers, Dirk

from docker-security.

Hi Paolo,

Still can't detect the picture.

Maybe the picture path is wrong. I'll check it.

Did you get anywhere? Your fork needs a rebase to include the picture. But still if I copy locally my files over to your pulled repo, the PDF doesn't show the image in "001 - Threats.md".

Cheers, Dirk

from docker-security.

Hi @drwetter,
yes, I did (a minute ago :D). Please check my fork.

The file in the assets directory was moved into the src/images directory.
Other images and book related assets should be place there too.

Check the resulting PDF.

Please let me known whether you want to flatten the repository directory structure.

Cheers,
Paulo A. Silva

from docker-security.

Hi Paolo,

I see the picture now, thanks! And so fast ;-)

Please let me known whether you want to flatten the repository directory structure.

For the second edition the first structure you suggested (OWASP/API-Security) may sound ok (unless one doesn't want to use branches for older releases)

For the first edition: couldn't we just keep my structure, create a subdir for the PDF build and work with COPY commands from the main directory so that a user doesn't have to look where the right directory is where he/she has to go to?

Cheers, Dirk

from docker-security.

Hi @drwetter,

For the first edition: couldn't we just keep my structure, create a subdir for the PDF build and work with COPY commands from the main directory so that a user doesn't have to look where the right directory is where he/she has to go to?

Cheers, Dirk

Of course we can.
I'll re-arrange everything today.

Cheers,
Paulo A. Silva

from docker-security.

Hi @drwetter,
Please, check whether my fork now meets your expectations.

Cheers,
Paulo A. Silva

from docker-security.

Hi Paolo,

looks good, thank you! Let me check details later.

Cheers, Dirk

from docker-security.

Closing with a big thank you, Paolo!

OT: Are you around in Amsterdam?

from docker-security.

Closing with a big thank you, Paolo!

My please, @drwetter.

OT: Are you around in Amsterdam?

Nop, not this time but if you meet the OWASP API Security Top 10 guys, please send them my greetings: we have been working hard to make it happen.

Keep strong & safe,
Paulo A. Silva

from docker-security.