Code Monkey home page Code Monkey logo

docker-image's Introduction

docToolchain

doctoolchain

Note

Corona/Covid19 is taking its toll. Time is short in these times and issues are not worked on. That is why I (rdmueller) have just introduced sponsored issues with higher priority. The label should help to explain why some issues will be preferred over others. If you want to sponsor an issue, just use the sponsor link on top of this page.
Important

If you have used docToolchain in the past, please be aware that the master branch is undergoing some changes.

If you are looking for a fresh version and are willing to accept changes in the configuration, this ng branch is the right thing for you.

If you are looking for what you are used to, then download the V1.0.0 release

Build Status Backers on Open Collective Sponsors on Open Collective

create awesome docs!

docToolchain is an implementation of the docs-as-code approach for software architecture. The basis of docToolchain is the philosophy that software documentation should be treated in the same way as code together with the arc42 template for software architecture.

Overview2

Contributors

This project exists thanks to all the people who contribute!

contributors

See also the list of contributors from the docs.

Each contribution is highly valuable and appreciated, no matter how big it is - there are no small contributions. Even just fixing a typo or starring the project helps to promote it.

I decided to give people and companies a chance to support this project. This is done through https://opencollective.com - the same service which Asciidoctor uses.

Companies who use docToolchain

docToolchain is still one of the smaller open source projects, but some companies already fully embrace the docs-as-code approach through docToolchain. If your company uses docToolchain, then feel free to add your company’s logo through a pull request. Please state in the PR that we are allowed to display the logo within all projects websites and documentation.

TomTom
DB Systel
codecentric
INNOQ

Backers

Thank you to all our backers!

Become a Backer

Sponsors

Support us by becoming a sponsor. Your logo will show up here with a link to your website.

Sponsor

History of this project

I just found an older tweet which reminded me of the roots and beginning of this project. It all began with two important parts. A script called asciidoc2confluence and some scripts to export diagrams from enterprise architect.

The first one is linked in the tweet above and the first commit dates back to mid of 2015. Since then, many contributors have joined! Thank for all of your contributions!

docker-image's People

Contributors

ascheman avatar dbroeglin avatar ehmkah avatar ko avatar llois41 avatar pacovk avatar rdmueller avatar sebastianmattar avatar

Stargazers

avatar avatar

Watchers

avatar avatar avatar avatar

Forkers

dbroeglin ko ewoks tregin abellmann sabatmonk nicolaimainiero llois41 m-gora phertweck sebastianmattar ehmkah mh182 delgurth simoncw pacovk peni4142

docker-image's Issues

run as non-root

currently, the docker image creates files with root permissions.
It would be better if it would create files with the owner of the host system user.
It seems that you can run the image with the host system user by specifying -u $(id -u ${USER}):$(id -g ${USER}) but this user has problems with files within the container since these where created by root.

need a combination of docker build and local build

the docker build (as implemented for https://github.com/aim42/htmlSanityCheck) works quite nice.

A project which uses this build has as benefit nearly no docToolchain code under version control.

The major drawback is that it does not work in virtualized environments.

We need an approach where a shell script is able to switch between a docker build and a local build where only a reference to the repository (submodule?) is kept.

Build new docker image

As few new features recently landed to docToolchain project (after version 1.2.0), it would be good to build & publish new dockerimage of the newest version to be able to take full advantage of the new features

  1. tag new docToolchain version
  2. build new docker image
  3. publish new docker image to docker.hub

Unable to build docker image

what i did:
$ git checkout master
$ cd alpine
$ docker build . -t rdmueller/doctoolchain:master

error:
 => => transferring dockerfile: 1.52kB                                                                                          0.0s
 => [internal] load .dockerignore                                                                                               0.0s
 => => transferring context: 2B                                                                                                 0.0s
 => [internal] load metadata for docker.io/library/openjdk:14-jdk-alpine                                                        2.8s
 => [auth] library/openjdk:pull token for registry-1.docker.io                                                                  0.0s
 => CACHED [1/8] FROM docker.io/library/openjdk:14-jdk-alpine@sha256:b8082268ef46d44ec70fd5a64c71d445492941813ba9d68049be6e63a  0.0s
 => [2/8] RUN ECHO "add needed tools" &&     apk add --no-cache curl wget zip unzip git bash     git     graphviz     python  144.4s
 => ERROR [3/8] RUN    gem install pygments.rb                                                                                  1.8s
------
 > [3/8] RUN    gem install pygments.rb:
#7 1.800 ERROR:  While executing gem ... (ArgumentError)
#7 1.800     wrong number of arguments (given 4, expected 1)
------
executor failed running [/bin/sh -c gem install pygments.rb]: exit code: 1
 => => transferring dockerfile: 1.52kB                                                                                          0.0s
 => [internal] load .dockerignore                                                                                               0.0s
 => => transferring context: 2B                                                                                                 0.0s
 => [internal] load metadata for docker.io/library/openjdk:14-jdk-alpine                                                        2.8s
 => [auth] library/openjdk:pull token for registry-1.docker.io                                                                  0.0s
 => CACHED [1/8] FROM docker.io/library/openjdk:14-jdk-alpine@sha256:b8082268ef46d44ec70fd5a64c71d445492941813ba9d68049be6e63a  0.0s
 => [2/8] RUN ECHO "add needed tools" &&     apk add --no-cache curl wget zip unzip git bash     git     graphviz     python  144.4s
 => ERROR [3/8] RUN    gem install pygments.rb                                                                                  1.8s
------
 > [3/8] RUN    gem install pygments.rb:
#7 1.800 ERROR:  While executing gem ... (ArgumentError)
#7 1.800     wrong number of arguments (given 4, expected 1)
------
executor failed running [/bin/sh -c gem install pygments.rb]: exit code: 1```

Question: pandoc support

I see that packages for pandoc in alpine were added to the Dockerfile but commented out. Is there any rationale as to why this is the case? Unofficial support? Preference to download and install from cabal? Forgot to uncomment?

For context, I recently learned that I also need to generate .docx output for some documents, so I've been looking down that dependency tree lately. It would be great to continue to use this docToolchain docker image for all related workflows.

Custom repository or proxy requried

We want to use the new dtcw using the docker image. The docker image is downloaded und started just fine but the dependencies can't be resolved. This happens to due to a restricted internet access in our CI environment. There are a few possibilities I can think of

  1. Provide a proxy configuration inside the docker container which is managed as a volumn or environment variables
  2. We could create a custom child image which provide the proxy settings.
  3. Provide a custom repository setting where the dependencies are downloaded from
  4. Provide all dependencies with this docker image (#15)

The main issue is

docker-image/alpine/Dockerfile

Line 43 in 147cf90

./gradlew dependencies && \
I guess.

ancestorId is not taken into account

We have the docker container running and want to publish to confluence.

When I use the container to publish the pages they will always be created in the root hierarchy of the confluence space instead of under the ancestorId.

If I use doctoolchain directly after cloning the repository the publishToConfluence works fine and the pages will be created at the desired ancestor.

This is even logged by doctoolchain itself (notice the missing /pages/....)

image

image

ARM64 Support

Need support for ARM64. Current error:

WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested

I can't generate documents from Confluence.

Hello,

I'm trying to create AsciiDoc from Confluence on the company's intranet without a connection to the Internet.

For this I use the Docker image doctoolchain/doctoolchain-with-pandoc:v2.2.1.

Creating pages and deploying them to Confluence works without any problems.

Pandoc is not found when calling ~/docToolchain/dtcw exportConfluence.

I ran the image on my private PC. Even there he can't find pandoc. not even in the latest version.

Am I missing a configuration or is something else not right?

// for exportConfluence task
export = [
srcDir: 'sample_data',
destDir: 'src/docs2'
]

Both directories begin in the root directory of the project.

Best regards

Martin

Running the image fails because of deprecated gradle features

Running the command for windows (docker with Windows/WSL2) yields the following output:

c:\...\src>docker run --rm --entrypoint /bin/bash -it -v %cd%:/project doctoolchain/doctoolchain:v2.2.1 -c "doctoolchain . %1 %2 %3 %4 %5 %6 %7 %8 %9 -PinputPath=src/main/asciidoc -PmainConfigFile=config/docToolchain.groovy && exit"
Starting a Gradle Daemon (subsequent builds will be faster)

> Configure project :

Config file '/project/config/docToolchain.groovy' does not exist'
[ant:input]
[ant:input] do you want me to create a default one for you? (y, n)
y
You are using one or more deprecated Asciidoctor Gradle plugin features. To help with migration run with --warning-mode=all.

FAILURE: Build failed with an exception.

* Where:
Script '/home/dtcuser/docToolchain/scripts/AsciiDocBasics.gradle' line: 40

* What went wrong:
A problem occurred evaluating script.
> /project/config/docToolchain.groovy (No such file or directory)

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.2/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 21s

Jenkins Agent needs open file permissions

When running in Jenkins (cf. changes to docToolchain/ci-cd-demo#1) the jenkins-ssh-agent might be run by arbitrary users (e.g., Jenkins runs the Docker container with some arbitrary -u <user>:<group> flag). Therefore it is necessary to have write permissions to some global or Jenkins specific files and directories, i.e.,

  • /etc/environment
  • /home/jenkins/.gradle

inputPath in Config.groovy not taken into account by all tasks

It looks like changing inputPath in Config.groovy does not work. The value of inputPath defined in gradle.properties is still used by, at least, exportChangelog & exportContributors

So with a standard template downloaded and unzipped I get errors for exportChangelog because it still looks for src/docs. I can fix it with something like this in Config.groovy:

changelog.with {
    dir = 'src'
}

However, at that point, same issue with exportContributors.

The code for my test is in the first commit of this repository: https://github.com/picobank/pico-travel-docs/tree/80d25537b0d88e965b30a502c41df101739a57eb

I'm not blocked, I'm only just doing discovery around docToolchain and it was simpler to just move everything into src/docs as expected by the tools. But I thought I might give you some feedback.

:generateHTML NO-SOURCE

Build reports "successful", but no result files:

 ~/d/d/doc > doctoolchain.sh generateHTML                                                                                                                                     10.2s  Fr 26 Jul 2019 11:20:09 CEST
Starting a Gradle Daemon, 1 incompatible and 1 stopped Daemons could not be reused, use --status for details
:generateHTML NO-SOURCE

BUILD SUCCESSFUL in 8s

with doctoolchain.sh:

#!/usr/bin/env bash
#CONFIG=config/docToolchain.groovy
CONFIG=Config.groovy
docker run --rm -it --entrypoint /bin/bash -v ${PWD}:/project rdmueller/doctoolchain:v1.1.0 \
-c "doctoolchain . $1 $2 $3 $4 $5 $6 $7 $8 $9 -PmainConfigFile=$CONFIG && exit"

In contrast, when I start docToolchain from a local installation, it works:

cd /home/gm/dev/docToolchain/bin/ ; ./doctoolchain /home/gm/dev/docToolchain_sample/doc generateHTML

> Configure project :
index.adoc

> Task :generateHTML
unsupported Java version "11", defaulting to 1.7
Converting /home/gm/dev/docToolchain_sample/doc/index.adoc

BUILD SUCCESSFUL in 5s
1 actionable task: 1 executed

197 vulnerabilities detected in 3.1.2

Dear devs,

can you please consider using a dependency bot like renovate to update the dependencies?

Due to a corporate rule, I mirror the docker image on our internal registry. We are using Quay and it has an inbuilt security scanner. Unfortunately, there is no reporting feature, so I am pasting it here directly in. Please note that the report is based on the image tag v3.1.2. Quay fixes most of the dependencies automatically by introducing new docker layers with updated package versions, but I guess it will help everyone if you do the fixes directly in the image.

Summary

Quay Security Scanner has detected 197 vulnerabilities.
Patches are available for 193 vulnerabilities.
24 Critical-level vulnerabilities.
88 High-level vulnerabilities.
55 Medium-level vulnerabilities.
25 Low-level vulnerabilities.
5 Unknown-level vulnerabilities.

Attachment with details

Sec_report.pdf

dtc jenkins build agent(s) wanted

As a Jenkins user I would like to easily use docToolchain as Docker build agent in order to simplify my Jenkins configuration/setup.

Is the alpine image broken?

Just to confirm that I am not doing something wrong. For me it looks like the alpine image for doctoolchain is broken. Could someone confirm it?

If I build the standard doctoolchain with the following setup I receive an error, see below.

export DOCKERHUB_USERNAME=doctoolchainy
export DTC_VERSION=v2.0.5  


docker-image % docker build \
  -t ${DOCKERHUB_USERNAME}/doctoolchain:${DTC_VERSION} \
  --build-arg DTC_VERSION=${DTC_VERSION} \
  alpine

I see the following error.

=> ERROR [ 6/12] RUN gem update --system  --no-rdoc --no-ri                                                                                                      7.1s
------                                                                                                                                                                 
 > [ 6/12] RUN gem update --system  --no-rdoc --no-ri:                                                                                                                 
#10 6.979 ERROR:  Error installing rubygems-update:                                                                                                                    
#10 6.979       There are no versions of rubygems-update (= 3.4.6) compatible with your Ruby & RubyGems                                                                
#10 6.979       rubygems-update requires Ruby version >= 2.6.0. The current ruby version is 2.5.0.                                                                     
#10 6.979 ERROR:  While executing gem ... (NoMethodError)
#10 6.979     undefined method `version' for nil:NilClass
#10 6.998 Updating rubygems-update

Is it possible that the docker image for alpine is somehow broken?

Could not create parent directory for lock file when a non-root user

How to run the image using a non-root user offline?

The previously downloaded already locally available gradle distribution (downloaded at the moment of building the https://github.com/docToolchain/docker-image/blob/master/alpine/Dockerfile) should be used.

Problem

I integrated this docker-image in my Jenkins-CI pipeline like this.

pipeline {

    agent {
        docker {
            image 'rdmueller/doctoolchain:rc-1.2.0'
        }
    }

    stages {
        stage ("Generate PDFs") {
            steps {
                sh "doctoolchain . generatePDF"
            }
        }
}

Which is equivalent to

#!/usr/bin/env bash
docker run --user jenkinbuilduser --rm -it --entrypoint /bin/bash -v ${PWD}:/project rdmueller/doctoolchain:rc-1.2.0 \
-c "doctoolchain . $1 $2 $3 $4 $5 $6 $7 $8 $9 -PinputPath=src/main/asciidoc -PmainConfigFile=config/docToolchain.groovy && exit"

or

#!/usr/bin/env bash
docker run --user 501 --rm -it --entrypoint /bin/bash -v ${PWD}:/project rdmueller/doctoolchain:rc-1.2.0 \
-c "doctoolchain . $1 $2 $3 $4 $5 $6 $7 $8 $9 -PinputPath=src/main/asciidoc -PmainConfigFile=config/docToolchain.groovy && exit"

from your documentation. Note the added option --user jenkinbuilduser and --user 501.

But this leads to exceptions like

Exception in thread "main" java.lang.RuntimeException: Could not create parent directory for lock file /docToolchain/?/.gradle/wrapper/dists/gradle-6.0.1-bin/1lxlpkiy24sb18odw96cp4ojv/gradle-6.0.1-bin.zip.lck
	at org.gradle.wrapper.ExclusiveFileAccessManager.access(ExclusiveFileAccessManager.java:43)
	at org.gradle.wrapper.Install.createDist(Install.java:48)
	at org.gradle.wrapper.WrapperExecutor.execute(WrapperExecutor.java:107)
	at org.gradle.wrapper.GradleWrapperMain.main(GradleWrapperMain.java:63)

My investigations showed that /root/.gradle (which holds the downloaded gradle distribution created by https://github.com/docToolchain/docker-image/blob/master/alpine/Dockerfile#L41) is not accessible by a non-root user like jenkinbuilduser. gradlew interprets it that no distribution is locally available and starts the download of the gradle distribution into /docToolchain/?/.gradle. This directory is read-only, which leads to the exception.

First workaround (non-working)

My first workaround was to run the image as user root.

pipeline {

    agent {
        docker {
            image 'rdmueller/doctoolchain:rc-1.2.0'
            args """-u root """
        }
    }

    stages {
        stage ("Generate PDFs") {
            steps {
                sh "doctoolchain . generatePDF"
            }
        }
}

Which is equivalent to

#!/usr/bin/env bash
docker run --rm -it --user root --entrypoint /bin/bash -v ${PWD}:/project rdmueller/doctoolchain:rc-1.2.0 \
-c "doctoolchain . $1 $2 $3 $4 $5 $6 $7 $8 $9 -PinputPath=src/main/asciidoc -PmainConfigFile=config/docToolchain.groovy && exit"

(Note the option --user root)

This way there is no exception, the generation works, but the generated file are owned by user root on the host file-system. This is not acceptable, because these files cannot be removed anymore (at least not by a next build steps within the build job. It can only be deleted manually by a fellow administrator with root-user rights)

Second workaround (partially working)

Set a non-root user and a rw-directory for the gradle distribution.

#!/usr/bin/env bash
docker run --user 501 -e GRADLE_USER_HOME='/project/.gradlecustom' --rm -it ---entrypoint /bin/bash -v ${PWD}:/project rdmueller/doctoolchain:rc-1.2.0 \
-c "doctoolchain . $1 $2 $3 $4 $5 $6 $7 $8 $9 -PinputPath=src/main/asciidoc -PmainConfigFile=config/docToolchain.groovy && exit"

gradlew will download a distribution to /project/.gradlecustom , the generation succeeds and the generated files are owned by a non-root user.

The problem: A buildjob should not have access to the internet.

The remaining problem

The issue is still remaining: How to run the image using a non-root user offline. The previously downloaded already locally available gradle distribution should be used.

Further reduce the image size

A way of reducing the image size a bit more would be to delete the .git repositories after `docToolchain has been cloned. You could gain about 27Mb:

bash-4.4# du -ks $(find /docToolchain -name .git)
4	/docToolchain/resources/reveal.js/.git
4	/docToolchain/resources/asciidoctor-reveal.js/.git
26912	/docToolchain/.git

publishToConfluence does not respect preambleTitle

I changed the title to something else like "Architecture". Instead it still uses arc42.

confluence.with {
    input = [[ file: "build/html5/arc42-template.html", ancestorId: "46956627" ]]
    api = 'https://wiki.abc.de/rest/api/'
    spaceKey = 'ABC'
    createSubpages = false
    preambleTitle = 'Architektur' // This should be used
    pagePrefix = ''
    pageSuffix = ''
    credentials = "test:test".bytes.encodeBase64().toString()
    extraPageContent = 'test'
}

This is how i mount it

#!/usr/bin/env bash
#CONFIG=config/docToolchain.groovy
docker run --rm -it --entrypoint /bin/bash -v /home/user/projects/test/volume/htdocs:/project rdmueller/doctoolchain:v1.1.0 \
-c "doctoolchain . publishToConfluence --info -PmainConfigFile=./docs/Config.groovy -PinputPath=./docs && exit"

And this is the output from the cli

Putting task artifact state for task ':publishToConfluence' into context took 0.001 secs.
Up-to-date check for task ':publishToConfluence' took 0.0 secs. It is not up-to-date because:
Task has not declared any outputs.
docToolchain> docDir: /project
publish /project/build/html5/arc42-template.html
arc42

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.