eugene-khyst / letsencrypt-docker-compose Goto Github PK

Set up Nginx and Let’s Encrypt in less than 3 minutes with a Docker Compose project that automatically obtains and renews free Let's Encrypt SSL/TLS certificates and sets up HTTPS in Nginx for multiple domain names. Configuration is done using a simple CLI tool.

License: Apache License 2.0

Dockerfile 4.77% Shell 29.24% JavaScript 54.62% Handlebars 11.37%

certbot certbot-ssl certificate docker docker-compose https lets-encrypt letsencrypt letsencrypt-certificates letsencrypt-https nginx nginx-https php-fpm renewing-certificates ssl tls tls-certificates tls-encryption

letsencrypt-docker-compose's Introduction

Hi there 👋

My name is Eugene Khyst. I am a Principal Software Engineer and Software Architect and creator of ArtistAssistApp - the best painting and drawing assistant app 🎨🖌️

Favorite tech stack

Favorite methodologies

letsencrypt-docker-compose's People

Contributors

Stargazers

Watchers

Forkers

tkramarets py-foundation-group-sa pyfundation dzyk devgoon b01lers artempronevskiy avineshwar silverark bjfrocha arintriyanto tecnoplug blueprin4 gigiperih freemindtronic partsar kolyars formula1 satya-500 fishermans opencontext wbsouza alexm24 tabital0 mthamil107 krlex lynns-73 alnwoks bencyril klad obovtunov tung1404 bachdgvn joeyfenny deepmd-tools dzhan85 disarticulate maddy023 michiski vuurpyl soufiene-aissa groskopf stevewithington kariroman sutirath polyneme chinaver2002 moroz anhgeeky rafaelpalomar nos486 gibmax entersb buralex mulspace gnjuguna sburnett neilh-cogapp fhause5 chrisychen muralimopidevi gskorupa xnr13 radubalanpro lifcey verlor paulo1211 alff0x1f rctportal starascendin yiming0 ozitv maximenasso libresoftasia petervajdecka02947 blackbluegl cunhabarbosa wildcommunist hendrixnguyen n-andronopoulos iindyverse dcwproject chatwithrajaram mcardle devclinton corysullivan dhanifajar15 cmbaughman ivanberry ertugoguz egl-tech vahpetr kvgoud1188 florintar xvmvx fiadigifrank taliptayfur fredsluter vishalindev peterliapin

letsencrypt-docker-compose's Issues

nginx error

nginx: [emerg] open() "/etc/nginx/vhosts/"mydomain".conf" failed (2: No such file or directory) in /etc/nginx/sites/"mydomain".conf:27
nginx_1 | Checking configuration for "mydomain"
nginx_1 | Waiting for Let's Encrypt certificates for "mydomain"
nginx_1 | 2022/05/28 07:48:28 [emerg] 1#1: open() "/etc/nginx/vhosts/"mydomain".conf" failed (2: No such file or directory) in /etc/nginx/sites/"mydomain"

scripts don't handle the double-quotes for $DOMAINS

the definition in config.env requires multiple domains to be within double quotes. The scripts don't remove them before iterating; the scripts get quotes in the paths
"site1.com site2.com" gives us /var/html/"site1.conf

My hack to fix it:

certbot.sh

fixed_domains=$(echo $DOMAINS | sed  's/"//g')
domain_list=($fixed_domains)
# domain_list=($DOMAINS)

nginx.sh

fixed_domains=$(echo $DOMAINS | sed  's/"//g')
for domain in $fixed_domains; do

thanks for your work

Nginx config file manual edits get lost upon doing CLI changes

I realized that any manual edits of Nginx's config files are lost whenever I run docker compose run --rm cli to add/remove domains or force certificate refresh.

Not sure if there is a workaround for this.

Btw, your project is excellent! Kudos on a very well-thought out and practical execution of a common problem.

Installing/Enabling PHP Container

Hello again,

I'm currently having some trouble getting a PHP Docker container to function properly with the Let's Encrypt Docker Compose. I've been trying to troubleshoot the issue, but so far I haven't had any luck in resolving it. I suspect that the issue may be related to the configuration or network setup of the container on my part. More specifically, my containers seem to build fine, but I'm having trouble with certbot.

Here's my fork of the project if you want to take a look.

If you have any insights or suggestions on how to fix this, I would greatly appreciate your assistance. Thank you in advance!

executor failed running [/bin/sh -c apk add --no-cache bash]: exit code: 1

Starting from a fresh Linux install, the docker compose command of step 4 fails.

The build process of certbot service raises this error:
=> ERROR [evgeniy-khyst/certbot 2/4] RUN apk add --no-cache bash

I couldn't find a way to get around the error. Here is the complete log.

$ docker compose up -d --build
[+] Building 1.6s (27/29)
 => [evgeniy-khyst/nginx internal] load build definition from Dockerfile                                                                                                                       0.0s
 => => transferring dockerfile: 91B                                                                                                                                                            0.0s
 => [evgeniy-khyst/cron internal] load build definition from Dockerfile                                                                                                                        0.0s
 => => transferring dockerfile: 91B                                                                                                                                                            0.0s
 => [evgeniy-khyst/certbot internal] load build definition from Dockerfile                                                                                                                     0.0s
 => => transferring dockerfile: 91B                                                                                                                                                            0.0s
 => [evgeniy-khyst/nginx internal] load .dockerignore                                                                                                                                          0.0s
 => => transferring context: 2B                                                                                                                                                                0.0s
 => [evgeniy-khyst/cron internal] load .dockerignore                                                                                                                                           0.0s
 => => transferring context: 2B                                                                                                                                                                0.0s
 => [evgeniy-khyst/certbot internal] load .dockerignore                                                                                                                                        0.0s
 => => transferring context: 2B                                                                                                                                                                0.0s
 => [evgeniy-khyst/nginx internal] load metadata for docker.io/library/nginx:1.23-alpine                                                                                                       0.6s
 => [evgeniy-khyst/certbot internal] load metadata for docker.io/certbot/certbot:v1.29.0                                                                                                       0.6s
 => [evgeniy-khyst/cron internal] load metadata for docker.io/library/alpine:3.16                                                                                                              0.6s
 => [evgeniy-khyst/nginx 1/7] FROM docker.io/library/nginx:1.23-alpine@sha256:659610aadb34b7967dea7686926fdcf08d588a71c5121edb094ce0e4cdbc45e6                                                 0.0s
 => [evgeniy-khyst/nginx internal] load build context                                                                                                                                          0.0s
 => => transferring context: 547B                                                                                                                                                              0.0s
 => CACHED [evgeniy-khyst/nginx 2/7] RUN apk add --no-cache openssl                                                                                                                            0.0s
 => CACHED [evgeniy-khyst/nginx 3/7] COPY default.conf /etc/nginx/conf.d/                                                                                                                      0.0s
 => CACHED [evgeniy-khyst/nginx 4/7] COPY gzip.conf options-ssl-nginx.conf hsts.conf /etc/nginx/includes/                                                                                      0.0s
 => CACHED [evgeniy-khyst/nginx 5/7] COPY site.conf.tpl /customization/                                                                                                                        0.0s
 => CACHED [evgeniy-khyst/nginx 6/7] COPY nginx.sh /customization/                                                                                                                             0.0s
 => CACHED [evgeniy-khyst/nginx 7/7] RUN chmod +x /customization/nginx.sh                                                                                                                      0.0s
 => [evgeniy-khyst/cron] exporting to image                                                                                                                                                    0.1s
 => => exporting layers                                                                                                                                                                        0.0s
 => => writing image sha256:69c81e0257f83fb06ee9a5efd02ec3e6a2f08f317f56c706ed351002c03a3b7b                                                                                                   0.0s
 => => naming to docker.io/evgeniy-khyst/nginx                                                                                                                                                 0.0s
 => => writing image sha256:105657c567975455736bd1d06633a53c56c8def1683b14a58920bf5b7e291ca5                                                                                                   0.0s
 => => naming to docker.io/evgeniy-khyst/cron                                                                                                                                                  0.0s
 => [evgeniy-khyst/certbot internal] load build context                                                                                                                                        0.0s
 => => transferring context: 91B                                                                                                                                                               0.0s
 => CACHED [evgeniy-khyst/certbot 1/4] FROM docker.io/certbot/certbot:v1.29.0@sha256:904fd574583ed30b2ebd3e17a4ab953a69589e0d4860c3199d117ad1dd7a4e94                                          0.0s
 => [evgeniy-khyst/cron 1/5] FROM docker.io/library/alpine:3.16@sha256:b95359c2505145f16c6aa384f9cc74eeff78eb36d308ca4fd902eeeb0a0b161b                                                        0.0s
 => [evgeniy-khyst/cron internal] load build context                                                                                                                                           0.0s
 => => transferring context: 95B                                                                                                                                                               0.0s
 => ERROR [evgeniy-khyst/certbot 2/4] RUN apk add --no-cache bash                                                                                                                              0.5s
 => CACHED [evgeniy-khyst/cron 2/5] RUN apk update &&     apk add --no-cache docker-cli docker-cli-compose                                                                                     0.0s
 => CACHED [evgeniy-khyst/cron 3/5] COPY renew_certs.sh /etc/periodic/daily/renew_certs                                                                                                        0.0s
 => CACHED [evgeniy-khyst/cron 4/5] RUN chmod +x /etc/periodic/daily/renew_certs                                                                                                               0.0s
 => CACHED [evgeniy-khyst/cron 5/5] WORKDIR /workdir                                                                                                                                           0.0s
------
 > [evgeniy-khyst/certbot 2/4] RUN apk add --no-cache bash:
#0 0.245 exec /bin/sh: exec format error
------
failed to solve: executor failed running [/bin/sh -c apk add --no-cache bash]: exit code: 1

certbot container fails with an error

I tried to execute the script with the option Multiple Docker Compose projects, because on that server I have Keycloak and PostgreSQL running in other containers.

? What's your domain name (e.g. example.com)? xxx.xx.xx
? What's your email for registration and recovery contact? [email protected]
? Want to have 'www' subdomain (e.g. www.example.com)? No
? Want to obtain a test certificate from a staging server? Yes
? What is the RSA key size in bits? 4096
? How do you want to configure Nginx? Reverse proxy
? Does the upstream server run as a Docker container on the same host? Yes
? What is the address of the proxied server (e.g. example-backend:8080)? keycloak_demo:8080
? Enable WebSocket proxying? No
? Want to add another domain? No
? What is the DH parameters size in bits? 2048
? Use Gzip? No
? Are the entered data correct? Yes

However the certbot container fails to start.

dependency failed to start: container letsencrypt-docker-compose-certbot-1 is unhealthy

Question about configuration and upstream

Hi! First of all, thank you very much for this tool, I'm new to Docker and it's very enriching and it works perfectly as I wanted! 😃

I would just like to understand better this part of the documentation :

https://github.com/eugene-khyst/letsencrypt-docker-compose#2-3-2-2

I have a Nuxt 3 application in the example folder : /home/docker/letsencrypt-docker-compose/examples/docker-nuxt
With a docker-compose.yml file.

But should I still add this (see below) in the docker-compose.yml at the root (/home/docker/letsencrypt-docker-compose) :

services:
  docker-nuxt:
    build: ./examples/docker-nuxt
    image: docker-nuxt
    restart: unless-stopped

Or do I just need to edit the docker-compose.yml from the example app with this (and remove above conf) :

version: "3"

services:
  docker-nuxt:
    build: ./examples/docker-nuxt
    image: docker-nuxt
    networks:
      - letsencrypt-docker-compose
    restart: unless-stopped

networks:
  letsencrypt-docker-compose:
    name: letsencrypt-docker-compose_default
    external: true

It's just a little confusing for me 😸
Thanks!

Anyway to loop over a list in nginx?

This is a great example but I want a more generic version. I'm currently rewriting it in nodejs but looking at the simplicity and elegance of nginx I'm somewhat disappointed I'm attempting to rewrite it. Nonetheless, I don't particularly want to have to add new servers everytime I want to update the conf. I'd prefer it loop yml file or something like that.

What I thought you did great

Allow the person to set multiple domains
Allow the person to set multiple emails
Handling the acme challenge
redirecting all other traffic to https
You're including both the toplevel domain and it's www subdomain for both nginx and the certbot
You've got a nice little cron job going in the background

To Make it more Generic

How do you load an array into nginx?
- I see an example of looping over a list
- But How do you load it in from environment variables or a file?
- This seems interesting but it's a little hard to understand for me
dynamically creating the file may work but that seems a bit rediculous
- This looks promising though
creating a bunch of child processes listening on the same port but then I would have a proxy server in front of my proxy server

Some features I am attempting to implement in nodejs

Large Numbers of domains (Restricted by safe numbers, lets encrypt and hardware, maybe more)
Large Numbers subdomains per domain (Restricted by safe numbers, lets encrypt and hardware, maybe more)
- catches invalid subdomains like bad wildcards or empty parts
- does not catch things like me.formula1.com vs *.formula1.com. No need for the me.formula1.com since the wildcard is catching it
  - update - it now does
Default email parameter
- this way the user doesn't have to set an email per virtual host
forwarding the request to another server instead of serving static files
- I don't know what a single server is going to be. Perhaps its just static files. But that can be it's own docker container
Ability to manually run it as a command and manually set certain args to override the config file
- https://www.npmjs.com/package/commander

Probably Going to ignore some things

Uppercase vs Lowercase - just sounds like a pain in the butt. I suppose I could just make everything lowercase though
- update - all domains and subdomains are now lowercase

I was going to setup a command to restart the server but I realized the cronjob will probably be fine

But I am a bit dissappointed though. Nginx seems like such a useful tool and nice on the eyes as well. And while you can do some amazing things in node, I'd prefer to have a simple configuration rather than big project I have to maintain.

Support HTTP2 / Support IPv6

Hi,

Thank you for your works, it's save time and it's cleaner!

There is a reason to not add per default HTTP2 support for TLS ?
Your nginx default configuration doesn't support IPv6
I've noticed that on testing the TLS grade on SSLLabs test service.
So, I wanted to add IPv6 support on nginx/site.conf.tpl :

Before:

(...)
listen 80;
(...)
listen 443 ssl;
(...)

After

(...)
listen 80;
listen [::]:80;
(...)
listen 443 ssl http2;
listen [::]:443 ssl http2;
(...)

The syntax is ok, I've done a reload without success, and so a total down/up of the services, but doesn't seems to be enough.
May be when the TLS certificates are done per Let's Encrypt it's only done on IPv4 if it's not respond on IPv6 and it's now too late to redo it, or may be it's not related to Let's Encrypt and more related to the nginx/docker config?

If you have any clue to help me to add support to IPv6 of my TLS certs.
Thank you!

upstream module

Everything is working as expected, thank you.
Now I would like to know how I could add the upstream module?
Like:
upstream backend {
server backend1.example.com weight=5;
server backend2.example.com:8080;
server unix:/tmp/backend3;

server backup1.example.com:8080   backup;
server backup2.example.com:8080   backup;

}

Certificate not valid

Hello! Thanks a lot for letsencrypt-docker-compose!
I try letsencrypt-docker-compose and get error "Certificate not valid"

Log docker-compose

Attaching to evgeniy-khyst-certbot-1, evgeniy-khyst-cron-1, evgeniy-khyst-nginx-1
evgeniy-khyst-nginx-1    | Checking configuration for jenkins.51.250.26.90.sslip.io
evgeniy-khyst-nginx-1    | Waiting for Let's Encrypt certificates for jenkins.51.250.26.90.sslip.io
evgeniy-khyst-nginx-1    | 2022/07/22 16:14:21 [notice] 1#1: using the "epoll" event method
evgeniy-khyst-nginx-1    | 2022/07/22 16:14:21 [notice] 1#1: nginx/1.23.1
evgeniy-khyst-nginx-1    | 2022/07/22 16:14:21 [notice] 1#1: built by gcc 11.2.1 20220219 (Alpine 11.2.1_git20220219) 
evgeniy-khyst-nginx-1    | 2022/07/22 16:14:21 [notice] 1#1: OS: Linux 5.4.0-122-generic
evgeniy-khyst-nginx-1    | 2022/07/22 16:14:21 [notice] 1#1: getrlimit(RLIMIT_NOFILE): 1048576:1048576
evgeniy-khyst-nginx-1    | 2022/07/22 16:14:21 [notice] 1#1: start worker processes
evgeniy-khyst-nginx-1    | 2022/07/22 16:14:21 [notice] 1#1: start worker process 13
evgeniy-khyst-nginx-1    | 2022/07/22 16:14:21 [notice] 1#1: start worker process 14
evgeniy-khyst-certbot-1  | Obtaining the certificate for jenkins.51.250.26.90.sslip.io with email [email protected]
evgeniy-khyst-certbot-1  | Saving debug log to /var/log/letsencrypt/letsencrypt.log
evgeniy-khyst-certbot-1  | Plugins selected: Authenticator webroot, Installer None
evgeniy-khyst-certbot-1  | Account registered.
evgeniy-khyst-certbot-1  | Requesting a certificate for jenkins.51.250.26.90.sslip.io and www.jenkins.51.250.26.90.sslip.io
evgeniy-khyst-certbot-1  | Performing the following challenges:
evgeniy-khyst-certbot-1  | http-01 challenge for jenkins.51.250.26.90.sslip.io
evgeniy-khyst-certbot-1  | http-01 challenge for www.jenkins.51.250.26.90.sslip.io
evgeniy-khyst-certbot-1  | Using the webroot path /var/www/certbot/jenkins.51.250.26.90.sslip.io for all unmatched domains.
evgeniy-khyst-certbot-1  | Waiting for verification...
evgeniy-khyst-nginx-1    | 54.93.230.64 - - [22/Jul/2022:16:14:25 +0000] "GET /.well-known/acme-challenge/sG9nIP8fQNdmtzT0U8Ad4CFx-sdVhBgbCGE5NEtbY8M HTTP/1.1" 200 87 "-" "Mozilla/5.0 (compatible; Let's Encrypt validation server; +https://www.letsencrypt.org)" "-"
evgeniy-khyst-nginx-1    | 23.178.112.107 - - [22/Jul/2022:16:14:26 +0000] "GET /.well-known/acme-challenge/sG9nIP8fQNdmtzT0U8Ad4CFx-sdVhBgbCGE5NEtbY8M HTTP/1.1" 200 87 "-" "Mozilla/5.0 (compatible; Let's Encrypt validation server; +https://www.letsencrypt.org)" "-"
evgeniy-khyst-nginx-1    | 23.178.112.106 - - [22/Jul/2022:16:14:26 +0000] "GET /.well-known/acme-challenge/EUrhUEs5MX4j5mJIDGBUqBxRf8l-68Rxz9ah0enSJy8 HTTP/1.1" 200 87 "-" "Mozilla/5.0 (compatible; Let's Encrypt validation server; +https://www.letsencrypt.org)" "-"
evgeniy-khyst-nginx-1    | Waiting for Let's Encrypt certificates for jenkins.51.250.26.90.sslip.io
evgeniy-khyst-nginx-1    | Waiting for Let's Encrypt certificates for jenkins.51.250.26.90.sslip.io
evgeniy-khyst-nginx-1    | 54.93.230.64 - - [22/Jul/2022:16:14:35 +0000] "GET /.well-known/acme-challenge/EUrhUEs5MX4j5mJIDGBUqBxRf8l-68Rxz9ah0enSJy8 HTTP/1.1" 200 87 "-" "Mozilla/5.0 (compatible; Let's Encrypt validation server; +https://www.letsencrypt.org)" "-"
evgeniy-khyst-nginx-1    | 18.222.142.167 - - [22/Jul/2022:16:14:35 +0000] "GET /.well-known/acme-challenge/EUrhUEs5MX4j5mJIDGBUqBxRf8l-68Rxz9ah0enSJy8 HTTP/1.1" 200 87 "-" "Mozilla/5.0 (compatible; Let's Encrypt validation server; +https://www.letsencrypt.org)" "-"
evgeniy-khyst-nginx-1    | 18.222.142.167 - - [22/Jul/2022:16:14:35 +0000] "GET /.well-known/acme-challenge/sG9nIP8fQNdmtzT0U8Ad4CFx-sdVhBgbCGE5NEtbY8M HTTP/1.1" 200 87 "-" "Mozilla/5.0 (compatible; Let's Encrypt validation server; +https://www.letsencrypt.org)" "-"
evgeniy-khyst-nginx-1    | 18.236.113.29 - - [22/Jul/2022:16:14:35 +0000] "GET /.well-known/acme-challenge/EUrhUEs5MX4j5mJIDGBUqBxRf8l-68Rxz9ah0enSJy8 HTTP/1.1" 200 87 "-" "Mozilla/5.0 (compatible; Let's Encrypt validation server; +https://www.letsencrypt.org)" "-"
evgeniy-khyst-nginx-1    | 18.236.113.29 - - [22/Jul/2022:16:14:35 +0000] "GET /.well-known/acme-challenge/sG9nIP8fQNdmtzT0U8Ad4CFx-sdVhBgbCGE5NEtbY8M HTTP/1.1" 200 87 "-" "Mozilla/5.0 (compatible; Let's Encrypt validation server; +https://www.letsencrypt.org)" "-"
evgeniy-khyst-nginx-1    | Waiting for Let's Encrypt certificates for jenkins.51.250.26.90.sslip.io
evgeniy-khyst-certbot-1  | Cleaning up challenges
evgeniy-khyst-certbot-1  | 
evgeniy-khyst-certbot-1  | Successfully received certificate.
evgeniy-khyst-certbot-1  | Certificate is saved at: /etc/letsencrypt/live/jenkins.51.250.26.90.sslip.io/fullchain.pem
evgeniy-khyst-certbot-1  | Key is saved at:         /etc/letsencrypt/live/jenkins.51.250.26.90.sslip.io/privkey.pem
evgeniy-khyst-certbot-1  | This certificate expires on 2022-10-20.
evgeniy-khyst-certbot-1  | These files will be updated when the certificate renews.
evgeniy-khyst-certbot-1  | NEXT STEPS:
evgeniy-khyst-certbot-1  | - The certificate will need to be renewed before it expires. Certbot can automatically renew the certificate in the background, but you may need to take steps to enable that functionality. See https://certbot.org/renewal-setup for instructions.
evgeniy-khyst-certbot-1 exited with code 0
evgeniy-khyst-nginx-1    | Switching Nginx to use Let's Encrypt certificate for jenkins.51.250.26.90.sslip.io
evgeniy-khyst-nginx-1    | Reloading Nginx configuration

https://community.letsencrypt.org/t/staging-doctored-durian-root-ca-x3-is-expired-breaks-test-environment/145689

sjkp/letsencrypt-siteextension#393

https://medium.com/geekculture/will-you-be-impacted-by-letsencrypt-dst-root-ca-x3-expiration-d54a018df257

cannot load certificate "/etc/nginx/ssl/dummy/test1.devcomanda.com/fullchain.pem":

nginx_1    | Waiting for Let's Encrypt certificates for something.com
nginx_1    | 2022/02/25 16:39:15 [emerg] 1#1: cannot load certificate "/etc/nginx/ssl/dummy/test1.devcomanda.com/fullchain.pem": BIO_new_file() failed (SSL: error:02001002:system library:fopen:No such file or directory:fopen('/etc/nginx/ssl/dummy/test1.devcomanda.com/fullchain.pem','r') error:2006D080:BIO routines:BIO_new_file:no such file)
nginx_1    | nginx: [emerg] cannot load certificate "/etc/nginx/ssl/dummy/test1.devcomanda.com/fullchain.pem": BIO_new_file() failed (SSL: error:02001002:system library:fopen:No such file or directory:fopen('/etc/nginx/ssl/dummy/test1.devcomanda.com/fullchain.pem','r') error:2006D080:BIO routines:BIO_new_file:no such file)
devcomanda_nginx_1 exited with code 1

DOMAINS=something.com
[email protected]
CERTBOT_TEST_CERT=1
CERTBOT_RSA_KEY_SIZE=4096

How to get certificate for subdomain?

I have my frontend deployed via netlify and with https at my domain, mydomain.com. I would now like to use this project to deploy my backend on a separate VM under the subdomain api.mydomain.com, what is the configuration to do this? Currently, in the interactive prompts given by the CLI there is only the option to use a www subdomain and nothing else. Thanks

Default network is `letsencrypt-docker-compose_default` missing from docs

Seems it's worth mentioning in the docs that the default setup creates network letsencrypt-docker-compose_default for all its containers. This proves useful given that the initialization script offers for Reverse proxy the option to Proxy to a server defined in the same docker-compose.yml (y/n).

In my case the service to which the reverse proxy relays requests is in a separate docker-compose.yml and has its own network. But even if it doesn't have its own network, it's worth mentioning that in that case my own services should be configured to use the letsencrypt-docker-compose_default network.

Moving to new installation

I am using letsencrypt-docker-compose as a rev proxy to several docker containers. But now I want to move some of the containers to a new instance. How would I go about the installation of letsencrypt-docker-compose on the new instance?

Do I simply do setup cli.sh config from the start? When it asks about generating staging certs, there is no longer the need (already did this in the current instance), but I suppose it will fail to generate new production certs as they already exist.

Or is it better to simply copy over the relevant parts of the config.json and the relevant subfolders of nginx-conf/?

Waiting for Let's Encrypt certificate for <bad-domain-name>

I added a new service but I mistyped the domain name. Something like example.orgg instead of .org.

I realized it too late and re-run to remove it. Removed successfully (no traces in config.json or under nginx-conf/) but now I see in the logs of letsencrypt-docker-compose-nginx-1 constantly the message Waiting for Let's Encrypt certificate for example.orgg.

Something I can do to clean up so this stops?

Can't create new cert ( openssl: not found )

I added new domain to existing domains according the instruction:
https://github.com/evgeniy-khist/letsencrypt-docker-compose#adding-a-new-domain-to-a-running-solution

docker compose down
docker compose up -d
docker compose logs -f

Maybe I need to do something additional? For example to remove some volumes or smth else...

Use not only Official & Alpine image

If I use not alpine - there is no APK command in the image:

nginx/Dockerfile

FROM nginx:1.23-alpine

RUN apk add --no-cache openssl # <-- Here is an exception

apk: not found

If I use ranadeeppolavarapu/nginx-http3 for example, I get an exception:

certbot  | Waiting for verification...
nginx    | Waiting for Let's Encrypt certificates for kuma.MY-DOMAIN.COM
nginx    | 2022/07/14 18:22:21 [error] 25#25: *2 open() "/etc/nginx/html/.well-known/acme-challenge/FP7eJZjfBMkfNXBdaQShJzcr1Jf2QBev2U8c2IEl7nw" failed (2: No such file or directory), client: 66.133.109.36, server: localhost, request: "GET /.well-known/acme-challenge/FP7eJZjfBMkfNXBdaQShJzcr1Jf2QBev2U8c2IEl7nw HTTP/1.1", host: "MY-DOMAIN.COM"
nginx    | 66.133.109.36 - - [14/Jul/2022:18:22:21 +0200] "GET /.well-known/acme-challenge/FP7eJZjfBMkfNXBdaQShJzcr1Jf2QBev2U8c2IEl7nw HTTP/1.1" 404 153 "-" "Mozilla/5.0 (compatible; Let's Encrypt validation server; +https://www.letsencrypt.org)"
nginx    | 66.133.109.36 - - [14/Jul/2022:18:22:21 +0200] "GET /.well-known/acme-challenge/ZC5_oGj8VwHaP2jzk-yPuEkiKRLNhnY-OiG2HWXyxGA HTTP/1.1" 404 153 "-" "Mozilla/5.0 (compatible; Let's Encrypt validation server; +https://www.letsencrypt.org)"
nginx    | 2022/07/14 18:22:21 [error] 25#25: *3 open() "/etc/nginx/html/.well-known/acme-challenge/ZC5_oGj8VwHaP2jzk-yPuEkiKRLNhnY-OiG2HWXyxGA" failed (2: No such file or directory), client: 66.133.109.36, server: localhost, request: "GET /.well-known/acme-challenge/ZC5_oGj8VwHaP2jzk-yPuEkiKRLNhnY-OiG2HWXyxGA HTTP/1.1", host: "www.MY-DOMAIN.COM"
certbot  | Challenge failed for domain MY-DOMAIN.COM
certbot  | Challenge failed for domain www.MY-DOMAIN.COM
certbot  | http-01 challenge for MY-DOMAIN.COM
certbot  | http-01 challenge for www.MY-DOMAIN.COM
certbot  | 
certbot  | Certbot failed to authenticate some domains (authenticator: webroot). The Certificate Authority reported these problems:
certbot  |   Domain: MY-DOMAIN.COM
certbot  |   Type:   unauthorized
certbot  |   Detail: ***.196.21.***: Invalid response from http://MY-DOMAIN.COM/.well-known/acme-challenge/FP7eJZjfBMkfNXBdaQShJzcr1Jf2QBev2U8c2IEl7nw: 404
certbot  | 
certbot  |   Domain: www.MY-DOMAIN.COM
certbot  |   Type:   unauthorized
certbot  |   Detail: ***.196.21.***: Invalid response from http://www.MY-DOMAIN.COM/.well-known/acme-challenge/ZC5_oGj8VwHaP2jzk-yPuEkiKRLNhnY-OiG2HWXyxGA: 404
certbot  | 
certbot  | Hint: The Certificate Authority failed to download the temporary challenge files created by Certbot. Ensure that the listed domains serve their content from the provided --webroot-path/-w and that files created there can be downloaded from the internet.

can't open '/var/log/letsencrypt/letsencrypt.log': No such file or directory

Upon starting services with cli.sh up, I see in logs:

letsencrypt-docker-compose-certbot-1  | tail: can't open '/var/log/letsencrypt/letsencrypt.log': No such file or directory

Doesn't seem to be fatal, but doesn't look too healthy either. Did I skip a step somewhere?

Crond not running properly

In my fork I had to change the last line of cron/Dockerfile to:

CMD ["crond", "-f", "-d", "0", "-c", "/etc/crontabs/"]

This makes the crond to read the crontabs properly and the certs get renewed automatically.

502 Bad Gateway or Connection refused while connecting to UPSTREAM

Hi there. I am trying to implement this helpful project but I got stuck :-(.
I think I made a mistake somewhere.
All certs were generated successfully, but I get 502 Bad Gateway.

docker logs:

╰─$ docker-compose logs nginx
nginx  | Checking configuration for MY-DOMAIN
nginx  | Checking configuration for kuma.MY-DOMAIN

errors log:

╰─$ tail  ./logs/error.log
2022/07/16 12:27:01 [notice] 1#1: start worker processes
2022/07/16 12:27:01 [notice] 1#1: start worker process 12
2022/07/16 12:27:01 [notice] 1#1: start worker process 13
2022/07/16 12:27:01 [notice] 1#1: start worker process 14
2022/07/16 12:27:01 [notice] 1#1: start worker process 15
2022/07/16 12:27:01 [notice] 1#1: start worker process 16
2022/07/16 12:27:01 [notice] 1#1: start worker process 17
2022/07/16 12:27:08 [error] 13#13: *1 connect() failed (111: Connection refused) while connecting to upstream, client: **.247.---.53, server: MY-DOMAIN, request: "GET / HTTP/1.1", upstream: "http://172.20.0.2:80/", host: "MY-DOMAIN"
2022/07/16 12:27:08 [error] 13#13: *1 connect() failed (111: Connection refused) while connecting to upstream, client: **.247.---.53, server: MY-DOMAIN, request: "GET /favicon.ico HTTP/1.1", upstream: "http://172.20.0.2:80/favicon.ico", host: "MY-DOMAIN", referrer: "https://MY-DOMAIN/"
2022/07/16 12:27:44 [error] 14#14: *4 connect() failed (111: Connection refused) while connecting to upstream, client: 159.**.173.198, server: MY-DOMAIN, request: "GET / HTTP/1.1", upstream: "http://172.20.0.2:80/", host: "185.196.21.187"

access logs:

╰─$ tail  ./logs/access.log
45.132.227.7 - - [16/Jul/2022:12:12:23 +0200] "GET / HTTP/1.1" 502 157 "-" "python-requests/2.22.0" "-"
128.14.209.162 - - [16/Jul/2022:12:21:04 +0200] "GET /owa/ HTTP/1.1" 502 559 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.113 Safari/537.36" "-"
**.247.---.53 - - [16/Jul/2022:12:24:27 +0200] "GET / HTTP/1.1" 502 559 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/100.0.4**6.143 YaBrowser/22.5.0.1916 Yowser/2.5 Safari/537.36" "-"
**.247.---.53 - - [16/Jul/2022:12:24:27 +0200] "GET /favicon.ico HTTP/1.1" 502 559 "https://MY-DOMAIN/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/100.0.4**6.143 YaBrowser/22.5.0.1916 Yowser/2.5 Safari/537.36" "-"
**.247.---.53 - - [16/Jul/2022:12:25:11 +0200] "GET / HTTP/1.1" 502 559 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/100.0.4**6.143 YaBrowser/22.5.0.1916 Yowser/2.5 Safari/537.36" "-"
**.247.---.53 - - [16/Jul/2022:12:25:12 +0200] "GET /favicon.ico HTTP/1.1" 502 559 "https://MY-DOMAIN/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/100.0.4**6.143 YaBrowser/22.5.0.1916 Yowser/2.5 Safari/537.36" "-"
193.118.53.194 - - [16/Jul/2022:12:25:38 +0200] "GET /owa/ HTTP/1.1" 502 559 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.113 Safari/537.36" "-"
**.247.---.53 - - [16/Jul/2022:12:27:08 +0200] "GET / HTTP/1.1" 502 559 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/100.0.4**6.143 YaBrowser/22.5.0.1916 Yowser/2.5 Safari/537.36" "-"
**.247.---.53 - - [16/Jul/2022:12:27:08 +0200] "GET /favicon.ico HTTP/1.1" 502 559 "https://MY-DOMAIN/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/100.0.4**6.143 YaBrowser/22.5.0.1916 Yowser/2.5 Safari/537.36" "-"
159.**.173.198 - - [16/Jul/2022:12:27:44 +0200] "GET / HTTP/1.1" 502 559 "-" "Mozilla/5.0 (Linux; Android 12; Pixel 4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/98.0.4758.101 Mobile Safari/537.36" "-"

Inside nginx container: cat /etc/nginx/sites:

And additional question: will it be helpful to ad here such a section:

    proxy_set_header Host                    $host;
    proxy_set_header X-Scheme                $scheme;
    proxy_set_header X-Forwarded-Proto       $scheme;
    proxy_set_header X-Auth-Request-Redirect $request_uri;
    proxy_set_header X-Real-IP               $remote_addr;
    proxy_set_header X-Forwarded-For         $proxy_add_x_forwarded_for;
    proxy_set_header Proxy                   "";

    add_header X-XSS-Protection              "1; mode=block";
    add_header X-Content-Type-Options        "nosniff";
    add_header X-Frame-Options               "deny";
    add_header Strict-Transport-Security     "max-age=31536000" always;
    add_header Alt-Svc                       'h3-29=":443"; ma=86400, h3=":443"; ma=86400;quic=":443"; ma=2592000; v="46,43"';
    resolver                                 8.8.4.4 8.8.8.8 valid=300s;

/docker-entrypoint.sh: exec: line 47: /customization/nginx.sh: not found

Hello,

First, thank you for your work!

I have been attempting to implement your patterns into my project but for some reason the nginx container is failing with the error:

I'm not sure if it's an error bubbling up from the shell script, but line 47 of nginx.sh seems relevant:

Might it be clear what could be causing this?

Thanks in advance.

Upstreams - where to put them?

Hi there ✌️ First of all, thank you for this project. I really need it.

I am trying to make it work with my domain + 1 subdomain.

I have configured A DNS record.
I have upstreams and don't have static HTML files (node js frontend) and I put them in nginx/default.conf:

server_names_hash_bucket_size 64;

include /etc/nginx/includes/gzip.conf;

include /etc/nginx/sites/*.conf;

upstream frontend {
  server frontend:3000;
}

upstream gateway {
  server gateway:8443;
}
...

My vhosts/my-domain.com.conf looks like this:

  location / {
    proxy_pass http://frontend;
  }

  location /api {
    rewrite /api/(.*) /$1  break;
    proxy_pass http://gateway/;
  }
...

When I run this I got an error:

[+] Running 3/3
 nginx    | Generating DH parameters, 2048 bit long safe prime, generator 2
nginx    | This is going to take a long time
certbot  | Waiting for nginx to start...
certbot  | Waiting for nginx to start...
certbot  | Waiting for nginx to start...
nginx    | ...................................................................................+.........................+...+...........................+......................................................................................................................................................................+....................................................................................................+......................................................................................................................+..................................................................................................................................................................+...+...............................................................................................................+................................................................+.......................................+.........................................+.....................................................+...................................+........+..............................................................................................+....+...................................................................................................................................................................................................................................................+...............+.............................................................................................+.+....................................+....................+............+.............................................................................................+..............................................+.................................................................................................................................................+.......................................................+.............................................................................................................................................+................+..................................+.......+...................................................+.....................+.........+..................+.........+.................................................+....................................................................................+.....................++*++*++*++*
nginx    | Checking configuration for MY-DOMAIN.COM
nginx    | Creating Nginx configuration file /etc/nginx/sites/MY-DOMAIN.COM.conf
nginx    | Generating dummy ceritificate for MY-DOMAIN.COM
nginx    | Generating a RSA private key
nginx    | .............................+++++
nginx    | ................................................................+++++
nginx    | writing new private key to '/etc/nginx/sites/ssl/dummy/MY-DOMAIN.COM/privkey.pem'
nginx    | -----
nginx    | Checking configuration for kuma.MY-DOMAIN.COM
nginx    | Creating Nginx configuration file /etc/nginx/sites/kuma.MY-DOMAIN.COM.conf
nginx    | Waiting for Let's Encrypt certificates for MY-DOMAIN.COM
nginx    | Generating dummy ceritificate for kuma.MY-DOMAIN.COM
nginx    | Generating a RSA private key
nginx    | ......................................................................................................................................................................................+++++
nginx    | ...........................................................................+++++
nginx    | writing new private key to '/etc/nginx/sites/ssl/dummy/kuma.MY-DOMAIN.COM/privkey.pem'
nginx    | -----
nginx    | Waiting for Let's Encrypt certificates for kuma.MY-DOMAIN.COM
certbot  | Obtaining the certificate for MY-DOMAIN.COM with email [email protected]
certbot  | Saving debug log to /var/log/letsencrypt/letsencrypt.log
certbot  | Plugins selected: Authenticator webroot, Installer None
nginx    | Waiting for Let's Encrypt certificates for MY-DOMAIN.COM
nginx    | Waiting for Let's Encrypt certificates for kuma.MY-DOMAIN.COM
certbot  | Account registered.
certbot  | Requesting a certificate for MY-DOMAIN.COM and www.MY-DOMAIN.COM
certbot  | Performing the following challenges:
certbot  | http-01 challenge for MY-DOMAIN.COM
certbot  | http-01 challenge for www.MY-DOMAIN.COM
certbot  | Using the webroot path /var/www/certbot/MY-DOMAIN.COM for all unmatched domains.
nginx    | Waiting for Let's Encrypt certificates for MY-DOMAIN.COM
certbot  | Waiting for verification...
nginx    | Waiting for Let's Encrypt certificates for kuma.MY-DOMAIN.COM
nginx    | 2022/07/14 18:22:21 [error] 25#25: *2 open() "/etc/nginx/html/.well-known/acme-challenge/FP7eJZjfBMkfNXBdaQShJzcr1Jf2QBev2U8c2IEl7nw" failed (2: No such file or directory), client: 66.133.109.36, server: localhost, request: "GET /.well-known/acme-challenge/FP7eJZjfBMkfNXBdaQShJzcr1Jf2QBev2U8c2IEl7nw HTTP/1.1", host: "MY-DOMAIN.COM"
nginx    | 66.133.109.36 - - [14/Jul/2022:18:22:21 +0200] "GET /.well-known/acme-challenge/FP7eJZjfBMkfNXBdaQShJzcr1Jf2QBev2U8c2IEl7nw HTTP/1.1" 404 153 "-" "Mozilla/5.0 (compatible; Let's Encrypt validation server; +https://www.letsencrypt.org)"
nginx    | 66.133.109.36 - - [14/Jul/2022:18:22:21 +0200] "GET /.well-known/acme-challenge/ZC5_oGj8VwHaP2jzk-yPuEkiKRLNhnY-OiG2HWXyxGA HTTP/1.1" 404 153 "-" "Mozilla/5.0 (compatible; Let's Encrypt validation server; +https://www.letsencrypt.org)"
nginx    | 2022/07/14 18:22:21 [error] 25#25: *3 open() "/etc/nginx/html/.well-known/acme-challenge/ZC5_oGj8VwHaP2jzk-yPuEkiKRLNhnY-OiG2HWXyxGA" failed (2: No such file or directory), client: 66.133.109.36, server: localhost, request: "GET /.well-known/acme-challenge/ZC5_oGj8VwHaP2jzk-yPuEkiKRLNhnY-OiG2HWXyxGA HTTP/1.1", host: "www.MY-DOMAIN.COM"
certbot  | Challenge failed for domain MY-DOMAIN.COM
certbot  | Challenge failed for domain www.MY-DOMAIN.COM
certbot  | http-01 challenge for MY-DOMAIN.COM
certbot  | http-01 challenge for www.MY-DOMAIN.COM
certbot  | 
certbot  | Certbot failed to authenticate some domains (authenticator: webroot). The Certificate Authority reported these problems:
certbot  |   Domain: MY-DOMAIN.COM
certbot  |   Type:   unauthorized
certbot  |   Detail: 185.196.21.187: Invalid response from http://MY-DOMAIN.COM/.well-known/acme-challenge/FP7eJZjfBMkfNXBdaQShJzcr1Jf2QBev2U8c2IEl7nw: 404
certbot  | 
certbot  |   Domain: www.MY-DOMAIN.COM
certbot  |   Type:   unauthorized
certbot  |   Detail: 185.196.21.187: Invalid response from http://www.MY-DOMAIN.COM/.well-known/acme-challenge/ZC5_oGj8VwHaP2jzk-yPuEkiKRLNhnY-OiG2HWXyxGA: 404
certbot  | 
certbot  | Hint: The Certificate Authority failed to download the temporary challenge files created by Certbot. Ensure that the listed domains serve their content from the provided --webroot-path/-w and that files created there can be downloaded from the internet.
certbot  | 
certbot  | Cleaning up challenges
certbot  | Some challenges have failed.
certbot  | Ask for help or search for solutions at https://community.letsencrypt.org. See the logfile /var/log/letsencrypt/letsencrypt.log or re-run Certbot with -v for more details.
certbot  | Obtaining the certificate for kuma.MY-DOMAIN.COM without email
certbot  | Saving debug log to /var/log/letsencrypt/letsencrypt.log
certbot  | Plugins selected: Authenticator webroot, Installer None
certbot  | Requesting a certificate for kuma.MY-DOMAIN.COM and www.kuma.MY-DOMAIN.COM
nginx    | Waiting for Let's Encrypt certificates for MY-DOMAIN.COM
certbot  | Performing the following challenges:
certbot  | http-01 challenge for kuma.MY-DOMAIN.COM
certbot  | http-01 challenge for www.kuma.MY-DOMAIN.COM
certbot  | Using the webroot path /var/www/certbot/kuma.MY-DOMAIN.COM for all unmatched domains.
nginx    | Waiting for Let's Encrypt certificates for kuma.MY-DOMAIN.COM
nginx    | 2022/07/14 18:22:26 [error] 25#25: *4 open() "/etc/nginx/html/.well-known/acme-challenge/ws0oOGqLomAntyC3ZxwDwgqi6TUQl_YEzypZam2p5aA" failed (2: No such file or directory), client: 54.93.239.45, server: localhost, request: "GET /.well-known/acme-challenge/ws0oOGqLomAntyC3ZxwDwgqi6TUQl_YEzypZam2p5aA HTTP/1.1", host: "kuma.MY-DOMAIN.COM"
nginx    | 54.93.239.45 - - [14/Jul/2022:18:22:26 +0200] "GET /.well-known/acme-challenge/ws0oOGqLomAntyC3ZxwDwgqi6TUQl_YEzypZam2p5aA HTTP/1.1" 404 153 "-" "Mozilla/5.0 (compatible; Let's Encrypt validation server; +https://www.letsencrypt.org)"
certbot  | Waiting for verification...
nginx    | 2022/07/14 18:22:26 [error] 25#25: *5 open() "/etc/nginx/html/.well-known/acme-challenge/ws0oOGqLomAntyC3ZxwDwgqi6TUQl_YEzypZam2p5aA" failed (2: No such file or directory), client: 66.133.109.36, server: localhost, request: "GET /.well-known/acme-challenge/ws0oOGqLomAntyC3ZxwDwgqi6TUQl_YEzypZam2p5aA HTTP/1.1", host: "kuma.MY-DOMAIN.COM"
nginx    | 66.133.109.36 - - [14/Jul/2022:18:22:26 +0200] "GET /.well-known/acme-challenge/ws0oOGqLomAntyC3ZxwDwgqi6TUQl_YEzypZam2p5aA HTTP/1.1" 404 153 "-" "Mozilla/5.0 (compatible; Let's Encrypt validation server; +https://www.letsencrypt.org)"
nginx    | 66.133.109.36 - - [14/Jul/2022:18:22:27 +0200] "GET /.well-known/acme-challenge/Cg-2ZvdeoPFYBXcw_9ElF2FSCQgtdwof7mAeFO-H4BY HTTP/1.1" 404 153 "-" "Mozilla/5.0 (compatible; Let's Encrypt validation server; +https://www.letsencrypt.org)"
nginx    | 2022/07/14 18:22:27 [error] 25#25: *6 open() "/etc/nginx/html/.well-known/acme-challenge/Cg-2ZvdeoPFYBXcw_9ElF2FSCQgtdwof7mAeFO-H4BY" failed (2: No such file or directory), client: 66.133.109.36, server: localhost, request: "GET /.well-known/acme-challenge/Cg-2ZvdeoPFYBXcw_9ElF2FSCQgtdwof7mAeFO-H4BY HTTP/1.1", host: "www.kuma.MY-DOMAIN.COM"
certbot  | Challenge failed for domain kuma.MY-DOMAIN.COM
certbot  | Challenge failed for domain www.kuma.MY-DOMAIN.COM
certbot  | http-01 challenge for kuma.MY-DOMAIN.COM
certbot  | http-01 challenge for www.kuma.MY-DOMAIN.COM
certbot  | 
certbot  | Certbot failed to authenticate some domains (authenticator: webroot). The Certificate Authority reported these problems:
certbot  |   Domain: kuma.MY-DOMAIN.COM
certbot  |   Type:   unauthorized
certbot  |   Detail: 185.196.21.187: Invalid response from http://kuma.MY-DOMAIN.COM/.well-known/acme-challenge/ws0oOGqLomAntyC3ZxwDwgqi6TUQl_YEzypZam2p5aA: 404
certbot  | 
certbot  |   Domain: www.kuma.MY-DOMAIN.COM
certbot  |   Type:   unauthorized
certbot  |   Detail: 185.196.21.187: Invalid response from http://www.kuma.MY-DOMAIN.COM/.well-known/acme-challenge/Cg-2ZvdeoPFYBXcw_9ElF2FSCQgtdwof7mAeFO-H4BY: 404
certbot  | 
certbot  | Hint: The Certificate Authority failed to download the temporary challenge files created by Certbot. Ensure that the listed domains serve their content from the provided --webroot-path/-w and that files created there can be downloaded from the internet.
certbot  | 
certbot  | Cleaning up challenges
certbot  | Some challenges have failed.
certbot  | Ask for help or search for solutions at https://community.letsencrypt.org. See the logfile /var/log/letsencrypt/letsencrypt.log or re-run Certbot with -v for more details.
certbot exited with code 0
nginx    | Waiting for Let's Encrypt certificates for MY-DOMAIN.COM
nginx    | Waiting for Let's Encrypt certificates for kuma.MY-DOMAIN.COM
nginx    | Waiting for Let's Encrypt certificates for MY-DOMAIN.COM
nginx    | Waiting for Let's Encrypt certificates for kuma.MY-DOMAIN.COM
nginx    | Waiting for Let's Encrypt certificates for MY-DOMAIN.COM
nginx    | Waiting for Let's Encrypt certificates for kuma.MY-DOMAIN.COM
nginx    | Waiting for Let's Encrypt certificates for MY-DOMAIN.COM
nginx    | Waiting for Let's Encrypt certificates for kuma.MY-DOMAIN.COM
nginx    | Waiting for Let's Encrypt certificates for MY-DOMAIN.COM
nginx    | Waiting for Let's Encrypt certificates for kuma.MY-DOMAIN.COM
nginx    | Waiting for Let's Encrypt certificates for MY-DOMAIN.COM
nginx    | Waiting for Let's Encrypt certificates for kuma.MY-DOMAIN.COM
....

My compose.yaml:

services:
  nginx:
    container_name: nginx
    build: ./nginx-config # I renamed this folder (it's ok)
    env_file: config.env
    ports:
      - 80:80
      - 443:443
    volumes:
      - static:/var/www/static:ro
      - nginx_conf:/etc/nginx/sites
      - letsencrypt_certs:/etc/letsencrypt
      - certbot_acme_challenge:/var/www/certbot
      - ./vhosts:/etc/nginx/vhosts
#      - ./html:/var/www/html. # <--- I don't have static webcontent. I have upstream (Maybe it should be /var/www/certbot?)
    restart: unless-stopped
    networks:
      - smz_default
      - nginx
      - monitoring

  certbot:
    build: ./certbot
#    image: evgeniy-khyst/certbot
    container_name: certbot
    env_file: config.env
    volumes:
      - letsencrypt_certs:/etc/letsencrypt
      - certbot_acme_challenge:/var/www/certbot
    networks:
      - nginx

  cron:
    build: ./cron
    container_name: cron
#    image: evgeniy-khyst/cron
    environment:
      COMPOSE_PROJECT_NAME: "${COMPOSE_PROJECT_NAME}"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - ./:/workdir:ro
    restart: unless-stopped
    networks:
      - nginx


networks:
  nginx:
    name: nginx
  smz_default:
    external: true
  monitoring:
    external: true

volumes:
  crowdsec-config:
  crowdsec-data:
  nginx_conf:
  letsencrypt_certs:
  certbot_acme_challenge:
  static:
    external: true
    name: smz_static

[emerg] 1#1: host not found in upstream

Thanks for this project!

in my case nginx won't start if an upstream container is not running.

I am attempting to use the nginx resolver directive to allow nginx to run if my upstream container is down.

Where would be the appropriate place to apply this nginx directive?

I have tried in my vhosts files...

location / {
    resolver 127.0.0.11 valid=30s ipv6=off;
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;
    proxy_pass http://my_docker_container:3000/;
}

and in site.conf.tpl...

resolver 127.0.0.11 valid=30s ipv6=off;

server {
    listen 80;
    server_name ${domain} www.${domain};

    location /.well-known/acme-challenge/ {
        root /var/www/certbot/${domain};
    }

    location / {
        return 301 https://$host$request_uri;
    }
}

server {
    listen 443 ssl;
    server_name ${domain} www.${domain};

    ssl_certificate /etc/nginx/sites/ssl/dummy/${domain}/fullchain.pem;
    ssl_certificate_key /etc/nginx/sites/ssl/dummy/${domain}/privkey.pem;

    include /etc/nginx/includes/options-ssl-nginx.conf;

    ssl_dhparam /etc/nginx/sites/ssl/ssl-dhparams.pem;

    include /etc/nginx/includes/hsts.conf;

    include /etc/nginx/vhosts/${domain}.conf;
}

Neither have worked.

html/${domain} directory not mounting static content to container (Solution: index.htm is case sensitive)

Hi, got everything working fine, however I'm having trouble getting my content on the container.
Before setup I had placed my static html files into /home/user/letsencrypt-docker-compose/html/mysite.com, and I have a valid index.html file inside.
When attempting to access my site I see it's directing traffic to the right place and the certificate is valid but I am met with a 403 Forbidden Error.
I checked docker compose logs and saw the following:
[error] 10#10: *51 "/var/www/html/**mysite.com**/index.html" is forbidden (13: Permission denied), client: **IP**, server: **DOMAIN**, request: "GET / HTTP/1.1", host: "**HOST**"

Based on my understanding, it appears that I may have placed my static content in an incorrect location or there may be some other issue that I am not aware of. I would greatly appreciate any guidance you may be able to provide on the matter. Thank you in advance for your assistance.

P.S. What would be the recommended way to update static content in the future?

CertBot can't handle multiple domains

The wrapper script seems to mash domains together given that the output of the jq call includes newlines, which you can't just feed into a shell array.

? Are the entered data correct? Yes
Writing config ./config.json
Compiling template ./templates/nginx.conf.hbs
Compiling template ./templates/servers.conf.hbs
Writing ./nginx-conf/nginx.conf
Writing ./nginx-conf/conf.d/domaina.com.conf
Writing ./nginx-conf/conf.d/siteb.org.conf
Executing command: docker compose exec --no-TTY nginx /letsencrypt-docker-compose/config-nginx.sh
2023/07/01 20:49:32 [notice] 2415#2415: signal process started

Configuring domains:
domaina.com
siteb.org
siteb.org
Configuring domain domaina.com
Switching Nginx to use Let's Encrypt certificate for domaina.com
Configuring domain siteb.org
Switching Nginx to use dummy certificate for siteb.org
Configuring domain siteb.org
Switching Nginx to use dummy certificate for siteb.org
Reloading Nginx configuration

Executing command: docker compose exec --no-TTY certbot /letsencrypt-docker-compose/certbot-certonly.sh
usage:
  certbot [SUBCOMMAND] [options] [-d DOMAIN] [-d DOMAIN] ...

Certbot can obtain and install HTTPS/TLS/SSL certificates.  By default,
it will attempt to use a webserver both for obtaining and installing the
certificate.
certbot: error: argument --rsa-key-size: invalid int value: '4096\n4096'
usage:
  certbot [SUBCOMMAND] [options] [-d DOMAIN] [-d DOMAIN] ...

Certbot can obtain and install HTTPS/TLS/SSL certificates.  By default,
it will attempt to use a webserver both for obtaining and installing the
certificate.
certbot: error: argument --rsa-key-size: invalid int value: '4096\n4096'

Obtaining the certificate for domain domaina.com
Let's Encrypt certificate for domaina.com already exists
Obtaining the certificate for domain siteb.org
Using email [email protected]
[email protected]
RSA key size is 4096
4096
Obtaining the certificate for domain siteb.org
Using email [email protected]
[email protected]
RSA key size is 4096
4096

Executing command: docker compose exec --no-TTY nginx /letsencrypt-docker-compose/config-nginx.sh
2023/07/01 20:49:35 [notice] 2441#2441: signal process started

Configuring domains:
domaina.com
siteb.org
siteb.org
Configuring domain domaina.com
Switching Nginx to use Let's Encrypt certificate for domaina.com
Configuring domain siteb.org
Switching Nginx to use dummy certificate for siteb.org
Configuring domain siteb.org
Switching Nginx to use dummy certificate for siteb.org
Reloading Nginx configuration

Changing the domains line to:

domains=$(jq -r '[.domains[].domain]|join(" ")' $config)

gives a properly output array

Non persistence of manually added setting in nginx-conf/conf.d/xxxx.conf

I've added manually in a specific config file under nginx-conf/conf.d/foo.mydonain.com.conf an entry:

       client_max_body_size 1000M;

And it worked after restarting. But it seems I have now lost the setting. The only significant action is I ran a cli.sh config to upgrade from a test certification to a production one.

Is it normal that this setting got lost?

All domains will apear offline when one backend container is offline at startup of nginx container

Nice work, it worked good!

Just one question.
Imagine, you have a server with two applications, one 'live production' version of a project and one development version of the same project. Each reachable by their own domain:
dev.myproject.com
live.myproject.com

Now when I start everything together (two backends for the application and one time letsencrypt-docker-compose) everything is nice.
When I stop the dev container (for some work on the docker config or whatever), the live application will still be reachable, great.
But then when I have the dev container stopped, and I restart the whole server (or the nginx container to be specific) then the nginx will not start anymore because it cannot find the development backend. It will fail with:

nginx_1    | 2022/11/12 09:50:55 [emerg] 1#1: host not found in upstream "dev_backend" in /etc/nginx/vhosts/dev.myproject.com.conf:6
nginx_1    | nginx: [emerg] host not found in upstream "dev_backend" in /etc/nginx/vhosts/dev.myproject.com.conf:6

Is there any way to have nginx start anyway with all backends that are available?

Add a domain to running config

I have setup a dual domain config according to your example. I would like to add a third domain now and I am not sure how to do that without breaking the running domains. How would I add an extra domain to a running config?

Removing a domain

Hi folks!

Thank you for sharing this amazing project.

When trying to remove a domain from my server I realized there was no explanation within the README file. It may be obvious for some people, but I think it would be nice to add a section to specify how to remove a domain. Something like :

Removing a domain name from a running solution

Remove the .conf file from vhost

Remove the domain name from config.env

docker compose down
docker volume rm nginx_conf
docker volume create --name=nginx_conf
docker compose up -d

Cron - auto renewal fails

Hello,
3 months ago, I installed Cerbot using this repo.
Everything worked fine; the dev and prod certificates were produced, and my website got to work with a good-working certificate.

3 month later.. The "cron" service tried to renew the certificate and failed with these logs: (I replaced the real domain with and the real IP with )

cron_1     | Renewing Let's Encrypt Certificates... (Wed Jan 11 02:00:00 UTC 2023)
cron_1     | 
cron_1     | - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
cron_1     | Processing /etc/letsencrypt/renewal/<DOMAIN>.conf
cron_1     | - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
cron_1     | Renewing an existing certificate for <DOMAIN> and www.<DOMAIN>
cron_1     | 
cron_1     | Certbot failed to authenticate some domains (authenticator: webroot). The Certificate Authority reported these problems:
cron_1     |   Domain: <DOMAIN>
cron_1     |   Type:   unauthorized
cron_1     |   Detail: <IP>: Invalid response from http://<DOMAIN>/.well-known/acme-challenge/ZCAkhfiArMRJF7uLvXSEftoDzF4eN24_TXrolUxiv2I: 404
cron_1     | 
cron_1     |   Domain: www.<DOMAIN>
cron_1     |   Type:   unauthorized
cron_1     |   Detail: <IP>: Invalid response from http://www.<DOMAIN>/.well-known/acme-challenge/h2aqQxpCb11Pw5qoDa1J-pZ6biPTECxXtPs_LkbCt_M: 404
cron_1     | 
cron_1     | Hint: The Certificate Authority failed to download the temporary challenge files created by Certbot. Ensure that the listed domains serve their content from the provided --webroot-path/-w and that files created there can be downloaded from the internet.
cron_1     | 
cron_1     | Saving debug log to /var/log/letsencrypt/letsencrypt.log
cron_1     | Failed to renew certificate <DOMAIN> with error: Some challenges have failed.
cron_1     | 
cron_1     | - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
cron_1     | All renewals failed. The following certificates could not be renewed:
cron_1     |   /etc/letsencrypt/live/<DOMAIN>/fullchain.pem (failure)
cron_1     | - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
cron_1     | 1 renew failure(s), 0 parse failure(s)
cron_1     | Ask for help or search for solutions at https://community.letsencrypt.org. See the logfile /var/log/letsencrypt/letsencrypt.log or re-run Certbot with -v for more details.
cron_1     | Reloading Nginx configuration
cron_1     | service "nginx" is not running container #1
cron_1     | run-parts: /etc/periodic/daily/renew_certs: exit status 1
cron_1     | Ren

Basically, it says that when requesting .well-known/acme-challenge/h2aqQxpCb11Pw5qoDa1J-pZ6biPTECxXtPs_LkbCt_M it gets a 404.
When I was checking the .well-known/acme-challenge// it did seem empty.
So I manually tried to add a file there, and it did succeed getting it with curl/postman.

Just to make it work again, I deleted all the volumes and reran the project again. But it will probably happen again every 3 months.

Any idea?
** I am using the latest version of this code

Thanks!

Nginx Container keeps crashing

Today restarted my service from scratch and now my nginx container is giving this:

Can't open /etc/letsencrypt/nginx/ssl/ssl-dhparams.pem for writing, No such file or directory
140237503966536:error:02001002:system library:fopen:No such file or directory:crypto/bio/bss_file.c:69:fopen('/etc/letsencrypt/nginx/ssl/ssl-dhparams.pem','w')
140237503966536:error:2006D080:BIO routines:BIO_new_file:no such file:crypto/bio/bss_file.c:76:
Can't open /etc/letsencrypt/nginx/ssl/ssl-dhparams.pem for writing, No such file or directory
140701478190408:error:02001002:system library:fopen:No such file or directory:crypto/bio/bss_file.c:69:fopen('/etc/letsencrypt/nginx/ssl/ssl-dhparams.pem','w')
140701478190408:error:2006D080:BIO routines:BIO_new_file:no such file:crypto/bio/bss_file.c:76:
Can't open /etc/letsencrypt/nginx/ssl/ssl-dhparams.pem for writing, No such file or directory
140199424687432:error:02001002:system library:fopen:No such file or directory:crypto/bio/bss_file.c:69:fopen('/etc/letsencrypt/nginx/ssl/ssl-dhparams.pem','w')
140199424687432:error:2006D080:BIO routines:BIO_new_file:no such file:crypto/bio/bss_file.c:76:
Can't open /etc/letsencrypt/nginx/ssl/ssl-dhparams.pem for writing, No such file or directory
140637844512072:error:02001002:system library:fopen:No such file or directory:crypto/bio/bss_file.c:69:fopen('/etc/letsencrypt/nginx/ssl/ssl-dhparams.pem','w')
140637844512072:error:2006D080:BIO routines:BIO_new_file:no such file:crypto/bio/bss_file.c:76:
Can't open /etc/letsencrypt/nginx/ssl/ssl-dhparams.pem for writing, No such file or directory
139887903821128:error:02001002:system library:fopen:No such file or directory:crypto/bio/bss_file.c:69:fopen('/etc/letsencrypt/nginx/ssl/ssl-dhparams.pem','w')
139887903821128:error:2006D080:BIO routines:BIO_new_file:no such file:crypto/bio/bss_file.c:76:
Can't open /etc/letsencrypt/nginx/ssl/ssl-dhparams.pem for writing, No such file or directory
140550094232904:error:02001002:system library:fopen:No such file or directory:crypto/bio/bss_file.c:69:fopen('/etc/letsencrypt/nginx/ssl/ssl-dhparams.pem','w')
140550094232904:error:2006D080:BIO routines:BIO_new_file:no such file:crypto/bio/bss_file.c:76:
Can't open /etc/letsencrypt/nginx/ssl/ssl-dhparams.pem for writing, No such file or directory
140694237838664:error:02001002:system library:fopen:No such file or directory:crypto/bio/bss_file.c:69:fopen('/etc/letsencrypt/nginx/ssl/ssl-dhparams.pem','w')
140694237838664:error:2006D080:BIO routines:BIO_new_file:no such file:crypto/bio/bss_file.c:76:
Can't open /etc/letsencrypt/nginx/ssl/ssl-dhparams.pem for writing, No such file or directory
140195733994824:error:02001002:system library:fopen:No such file or directory:crypto/bio/bss_file.c:69:fopen('/etc/letsencrypt/nginx/ssl/ssl-dhparams.pem','w')
140195733994824:error:2006D080:BIO routines:BIO_new_file:no such file:crypto/bio/bss_file.c:76:
Can't open /etc/letsencrypt/nginx/ssl/ssl-dhparams.pem for writing, No such file or directory
140138018372936:error:02001002:system library:fopen:No such file or directory:crypto/bio/bss_file.c:69:fopen('/etc/letsencrypt/nginx/ssl/ssl-dhparams.pem','w')
140138018372936:error:2006D080:BIO routines:BIO_new_file:no such file:crypto/bio/bss_file.c:76:
Can't open /etc/letsencrypt/nginx/ssl/ssl-dhparams.pem for writing, No such file or directory
140592796044616:error:02001002:system library:fopen:No such file or directory:crypto/bio/bss_file.c:69:fopen('/etc/letsencrypt/nginx/ssl/ssl-dhparams.pem','w')
140592796044616:error:2006D080:BIO routines:BIO_new_file:no such file:crypto/bio/bss_file.c:76:

How can I fix it? Or maybe try to force to recreate those files?

Changes to site.conf.tpl is not propergated

I have experienced issues with trying alternative nginx configuration via changes to site.conf.tpl. This was due to the fact that nginx.sh is only creating new /etc/nginx/sites/$domain.conf if it does not allready exist. This means that updated are not propergated to the docker volume.

Three of the created files owned by root?

After running docker compose run --rm cli I noticed that three of the generated files are owned by root:

./nginx-conf/conf.d/amazonsnitch.com.conf
./nginx-conf/nginx.conf
./config.json

Was it something I did? If not, shouldn't they all be editable? It's mentioned in the docs for example to edit ./nginx-conf/nginx.conf to make manual adjustmants to Nginx.

certbot and nginx waiting for eachother

Just created everything based on the directions, nginx_1 is periodically printing
Waiting for Let's Encrypt certificates for [mydomain]
and certbot_1 is printing
Waiting for nginx to start...
every now and then.

Did I mess up the config anywhere?