enigmabridge / certbot-external-auth Goto Github PK

Certbot external DNS, HTTP, TLSSNI domain validation plugin with JSON output and scriptable hooks, with Dehydrated compatibility

License: Other

Python 71.32% Shell 28.68%

certbot plugin json json-output configurator dns-validation

certbot-external-auth's People

Contributors

Stargazers

Watchers

Forkers

ph4r05 torxed carol8421 morristech c0degyver heikipikker bartzjegr gwg422 aronnovak 5l1v3r1 corbolais jetti888 sitedata zlaski jseparovic

certbot-external-auth's Issues

Not compatible with current version of certbot

The TLS-SNI challenge has been removed from the ACME standard. In certbot, the respective parts in the library seem to be gone as well.

certbot-external-auth, however, uses acme.challenges.TLSSNI01 in the line referenced below.

certbot-external-auth/certbot_external_auth/plugin.py

Line 219 in 07eca04

return [challenges.DNS01, challenges.HTTP01, challenges.TLSSNI01]

When certbot-external-auth is executed, we get the following error:
AttributeError: module 'acme.challenges' has no attribute 'TLSSNI01'

A solution would be to remove everything related to TLS-SNI.

Automatic renewal doesn't work

Hi,

I've managed to generate a certificate using certbot external auth:

certbot --staging -d xxx.yyy.zzz -a certbot-external-auth:out --certbot-external-auth:out-public-ip-logging-ok --preferred-challenges dns --certbot-external-auth:out-handler ./handler-example.sh certonly

The handler-example.sh generates an appropriate DNS entry and updates the DNS.

but when I try to renew:

certbot renew --force-renewal --cert-name xxx.yyy.zzz

I get the error:

Running manual mode non-interactively is not supported (yet)

Also, looking in /etc/letsencrypt/renewal/xxx.yyy.zzz, the config file doesn't have any reference to the external handler script I'm using to populate the DNS.

Any suggestions as to how I can get this to work?

Thanks

Andy

Incorrect instructions in manual mode

I am requesting a certificate for an internal domain using DNS validation. I am following the directions from [1] and [2].

My domain example.com is valid and available in the open, while the subdomain subdomain.example.com is reachable only if connected to the Intranet.

What I want is a certificate for subdomain.example.com.

Since the internal DNS is managed by our Network Office I have being using the manual mode.

If I use certbot-external-auth I get this:

$ sudo certbot --text \
                --agree-tos \
                --email [email protected] \
                --expand  \
                --renew-by-default \
               --configurator certbot-external-auth:out \
               --certbot-external-auth:out-text-mode \
               -d subdomain.example.com \
                  certonly
[sudo] password for user: 
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator certbot-external-auth:out, Installer certbot-external-auth:out
Obtaining a new certificate
/usr/lib/python3/dist-packages/josepy/jwa.py:107: CryptographyDeprecationWarning: signer and verifier have been deprecated. Please use sign and verify instead.
  signer = key.signer(self.padding, self.hash)
Performing the following challenges:
dns-01 challenge for subdomain.example.com

-------------------------------------------------------------------------------
NOTE: The IP of this machine will be publicly logged as having requested this
certificate. If you're running certbot in manual mode on a machine that is not
your server, please ensure you're okay with that.

Are you OK with your IP being logged?
-------------------------------------------------------------------------------
(Y)es/(N)o: Y
Please deploy a DNS TXT record under the name
subdomain.example.com with the following value:

aaa-aaa-aaa

Once this is deployed,
Press ENTER to continue
Waiting for verification...
Cleaning up challenges
Failed authorization procedure. subdomain.example.com (dns-01): urn:acme:error:connection :: The server could not connect to the client to verify the domain :: DNS problem: NXDOMAIN looking up 
TXT for _acme-challenge.subdomain.example.com

IMPORTANT NOTES:
 - The following errors were reported by the server:

   Domain: subdomain.example.com
   Type:   connection
   Detail: DNS problem: NXDOMAIN looking up TXT for
   _acme-challenge.subdomain.example.com

   To fix these errors, please make sure that your domain name was
   entered correctly and the DNS A/AAAA record(s) for that domain
   contain(s) the right IP address. Additionally, please check that
   your computer has a publicly routable IP address and that no
   firewalls are preventing the server from communicating with the
   client. If you're using the webroot plugin, you should also verify
   that you are serving files from the webroot path you provided.

the strange thing is that the instruction say to

deploy a DNS TXT record under the name subdomain.example.com

while in the end the DNS request is made to:

DNS problem: NXDOMAIN looking up TXT for  _acme-challenge.subdomain.example.com

If I use the "vanilla" certbot I am able to get the certificate:

$ sudo certbot --text \
              --agree-tos \
              --email [email protected] \
              -d subdomain.example.com \
              --manual \
              --preferred-challenges dns \
              --expand \
              --renew-by-default \
              --manual-public-ip-logging-ok \
                  certonly
[sudo] password for user: 
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator manual, Installer None
Obtaining a new certificate
/usr/lib/python3/dist-packages/josepy/jwa.py:107: CryptographyDeprecationWarning: signer and verifier have been deprecated. Please use sign and verify instead.
  signer = key.signer(self.padding, self.hash)
Performing the following challenges:
dns-01 challenge for subdomain.example.com

-------------------------------------------------------------------------------
Please deploy a DNS TXT record under the name
_acme-challenge.subdomain.example.com with the following value:

aaa-aaa-aaa

Before continuing, verify the record is deployed.
-------------------------------------------------------------------------------
Press Enter to Continue
Waiting for verification...
Cleaning up challenges

IMPORTANT NOTES:
 - Congratulations! Your certificate and chain have been saved at:
   /etc/letsencrypt/live/subdomain.example.com/fullchain.pem
   Your key file has been saved at:
   /etc/letsencrypt/live/subdomain.example.com/privkey.pem
   Your cert will expire on 2018-07-08. To obtain a new or tweaked
   version of this certificate in the future, simply run certbot
   again. To non-interactively renew *all* of your certificates, run
   "certbot renew"
 - If you like Certbot, please consider supporting our work by:

   Donating to ISRG / Let's Encrypt:   https://letsencrypt.org/donate
   Donating to EFF:                    https://eff.org/donate-le

and the instructions say to:

Please deploy a DNS TXT record under the name _acme-challenge.subdomain.example.com

with the domain request for the verification correclty made to _acme-challenge.subdomain.example.com.

So I think this a bug of certbot-external-auth.

certbot-external-auth not working with certbot 0.10

It appears that certbot.display.util.FileDisplay is changed in this revision which introduced an addition argument in it's constructor.

As as result line 165 of certbot_external_auth/plugin.py

displayer = display_util.FileDisplay(sys.stderr)

fails. I have to change it to

displayer = display_util.FileDisplay(sys.stderr, False)

to make it work in certbot 0.10

Question regarding usage with Ansible

This post mentions using Ansible together with JSON mode. But afaik Ansible can't talk to stdin like that, let alone in an asynchronous way. The problem I see is that the DNS TXT record creation (which Ansible has many modules for) needs to happen after the call to certbot, but while certbot is still running.
Is it maybe simply possible to execute the certbot command in multiple steps, i.e. first generate the token to stdout. Then run the validatation in a second step, after the DNS update has been made, and waited for?

ContextualVersionConflict

I have launched a fresh EC2 on Amazon - the Amazon Linux version.

sudo su
yum groupinstall 'Development Tools'
yum install gcc libffi-devel python-devel openssl-devel
pip install certbot
pip install certbot-external-auth
exit
certbot --text --agree-tos --email [email protected]
--expand --renew-by-default --configurator certbot-external-auth:out
--certbot-external-auth:out-public-ip-logging-ok
-d "bristol3.pki.enigmabridge.com" --preferred-challenges dns certonly

... and I got this error

Traceback (most recent call last):
File "/usr/local/bin/certbot", line 11, in
sys.exit(main())
File "/usr/local/lib/python2.7/site-packages/certbot/main.py", line 858, in main
plugins = plugins_disco.PluginsRegistry.find_all()
File "/usr/local/lib/python2.7/site-packages/certbot/plugins/disco.py", line 183, in find_all
plugin_ep = PluginEntryPoint(entry_point)
File "/usr/local/lib/python2.7/site-packages/certbot/plugins/disco.py", line 34, in init
self.plugin_cls = entry_point.load()
File "/usr/lib/python2.7/dist-packages/pkg_resources/init.py", line 2311, in load
self.require(*args, **kwargs)
File "/usr/lib/python2.7/dist-packages/pkg_resources/init.py", line 2328, in require
items = working_set.resolve(reqs, env, installer)
File "/usr/lib/python2.7/dist-packages/pkg_resources/init.py", line 812, in resolve
raise VersionConflict(dist, req).with_context(dependent_req)
ContextualVersionConflict: (six 1.8.0 (/usr/lib/python2.7/dist-packages), Requirement.parse('six>=1.9'), set(['mock']))

more info

Need more info on how to use this script. I am using the following command: certbot certonly --agree-tos -m [email protected] --renew-by-default --no-eff-email -d domain.com -d www.domain.com -a certbot-external-auth:out --certbot-external-auth:out-public-ip-logging-ok --preferred-challenges dns --certbot-external-auth:out-handler ./certbot-handler.sh. This results in what looks like information for three text records: token, validation, and key_auth (example below).

{"cmd": "perform_challenge", "type": "dns-01", "domain": "domain.com", "token": "Xdrw****lNdMtEj****wdLAT****_rIINO6K****wwg", "validation": "IfT2****3t8LP****4TeetW1****6pH2jY****zGgcc", "txt_domain": "_acme-challenge.domain.com", "key_auth": "Xdrw****lNdMtEj****wdLAT****_rIINO6K****wwg.KxGHb****AMkOo****G-****KwmVwC****euPm****s"}

Am I correct? If not which of these needs to be turned into a text record?

unable to renew the certs

I am not sure how to enable automatical renewal with external-auth so I am planning to use con job for automatic renewals. However, the renew command in authenticator mode is failing. Below is the command that I am using

certbot --text --agree-tos --email [email protected] --expand --renew-by-default -a certbot-external-auth:out --certbot-external-auth:out-public-ip-logging-ok --preferred-challenges dns --certbot-external-auth:out-handler ./le-route53.rb --certbot-external-auth:out-dehydrated-dns renew

I get the following error message

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Processing /etc/letsencrypt/renewal/wiki.x.y.z.conf
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Could not choose appropriate plugin: The certbot-external-auth:out plugin is not working; there may be problems with your existing configuration.
The error was: PluginError('Running manual mode non-interactively is not supported (yet)',)
Attempting to renew cert (wiki.x.y.z.com) from /etc/letsencrypt/renewal/wiki.x.y.z.conf produced an unexpected error: The certbot-external-auth:out plugin is not working; there may be problems with your existing configuration.
The error was: PluginError('Running manual mode non-interactively is not supported (yet)',). Skipping.

acme.jose replaced with josepy (library change upstream)

https://github.com/EnigmaBridge/certbot-external-auth/blob/master/certbot_external_auth/plugin.py#L33

from acme.jose import b64

needs to be replaced with

from josepy import b64

And the library to install is this one:

https://github.com/certbot/josepy

Because they're annoying bastards that change even their libraries mid way without any real notification. But hey, I'm just a developer.. What do I know.

(I'll submit a pull request if I got the time this week)

Execute permission on handler script

I received an obscure error from the following:

$ certbot --staging \
         --text --agree-tos --email [email protected] \
         --expand --renew-by-default \
         --configurator certbot-external-auth:out \
         --certbot-external-auth:out-public-ip-logging-ok \
         -d "example.com" \
         --preferred-challenges dns \
         --certbot-external-auth:out-handler /vagrant/handlers/dns.sh \
         certonly

Encountered exception during recovery
[Errno 13] Permission denied
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/certbot/error_handler.py", line 99, in _call_registered
    self.funcs[-1]()
  File "/usr/local/lib/python2.7/dist-packages/certbot/auth_handler.py", line 279, in _cleanup_challenges
    self.auth.cleanup(achalls)
  File "/usr/local/lib/python2.7/dist-packages/certbot_external_auth/plugin.py", line 289, in cleanup
    if self._is_classic_handler_mode() and self._call_handler("pre-cleanup") is None:
  File "/usr/local/lib/python2.7/dist-packages/certbot_external_auth/plugin.py", line 666, in _call_handler
    env=env)
  File "/usr/lib/python2.7/subprocess.py", line 711, in __init__
    errread, errwrite)
  File "/usr/lib/python2.7/subprocess.py", line 1343, in _execute_child
    raise child_exception
OSError: [Errno 13] Permission denied
An unexpected error occurred:
OSError: [Errno 13] Permission denied
Please see the logfiles in /var/log/letsencrypt for more details.
{"cmd": "report", "messages": []}

Initially I assumed it was demanding root/sudo, but that wasn't the case. It was actually complaining that there was no +x on the handler script. Can we get a more descriptive error?

The error message for a missing handler file could similarly be improved.

types.StringTypes deprecated

cur_record[FIELD_VALIDATION] = validation if isinstance(validation, types.StringTypes) else ''

in plugin.py, won't work because StringTypes is deprecated.

https://docs.python.org/3/library/types.html

AttributeError: module 'acme.challenges' has no attribute 'TLSSNI01'

Command:

certbot certonly --email [email protected] --configurator certbot-external-auth:out --certbot-external-auth:out-public-ip-logging-ok --agree-tos  --preferred-challenges dns  -d moo.novaak.net -d mah.novaak.net -d blah.novaak.net

Output:

Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator certbot-external-auth:out, Installer certbot-external-auth:out
Obtaining a new certificate
Performing the following challenges:
An unexpected error occurred:
AttributeError: module 'acme.challenges' has no attribute 'TLSSNI01'
Please see the logfiles in /var/log/letsencrypt for more details.
{"cmd": "report", "messages": []}

From the log:

2020-03-09 10:23:31,024:INFO:certbot._internal.auth_handler:Performing the following challenges:
2020-03-09 10:23:31,025:DEBUG:certbot._internal.log:Exiting abnormally:
Traceback (most recent call last):
  File "/usr/local/bin/certbot", line 10, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.7/site-packages/certbot/main.py", line 15, in main
    return internal_main.main(cli_args)
  File "/usr/local/lib/python3.7/site-packages/certbot/_internal/main.py", line 1347, in main
    return config.func(config, plugins)
  File "/usr/local/lib/python3.7/site-packages/certbot/_internal/main.py", line 1233, in certonly
    lineage = _get_and_save_cert(le_client, config, domains, certname, lineage)
  File "/usr/local/lib/python3.7/site-packages/certbot/_internal/main.py", line 121, in _get_and_save_cert
    lineage = le_client.obtain_and_enroll_certificate(domains, certname)
  File "/usr/local/lib/python3.7/site-packages/certbot/_internal/client.py", line 410, in obtain_and_enroll_certificate
    cert, chain, key, _ = self.obtain_certificate(domains)
  File "/usr/local/lib/python3.7/site-packages/certbot/_internal/client.py", line 344, in obtain_certificate
    orderr = self._get_order_and_authorizations(csr.data, self.config.allow_subset_of_names)
  File "/usr/local/lib/python3.7/site-packages/certbot/_internal/client.py", line 391, in _get_order_and_authorizations
    authzr = self.auth_handler.handle_authorizations(orderr, best_effort)
  File "/usr/local/lib/python3.7/site-packages/certbot/_internal/auth_handler.py", line 62, in handle_authorizations
    achalls = self._choose_challenges(authzrs)
  File "/usr/local/lib/python3.7/site-packages/certbot/_internal/auth_handler.py", line 206, in _choose_challenges
    self._get_chall_pref(authzr.body.identifier.value),
  File "/usr/local/lib/python3.7/site-packages/certbot/_internal/auth_handler.py", line 221, in _get_chall_pref
    plugin_pref = self.auth.get_chall_pref(domain)
  File "/usr/local/lib/python3.7/site-packages/certbot_external_auth/plugin.py", line 219, in get_chall_pref
    return [challenges.DNS01, challenges.HTTP01, challenges.TLSSNI01]
AttributeError: module 'acme.challenges' has no attribute 'TLSSNI01'
2020-03-09 10:23:31,028:ERROR:certbot._internal.log:An unexpected error occurred:

Versions:
certbot 1.3.0

certbot-external-auth 0.1.0

tokens that start with a dash break arguments passed to out-handler

Frequently when I try to do a cert renewal for multiple domains, the token the letsencrypt server gives me starts with a -. When this is the case, renewal fails.

I run certbot with these options:

/venv/bin/certbot \
        --server https://acme-v02.api.letsencrypt.org/directory \
        --text --agree-tos --email [email protected] \
        --expand --renew-by-default \
        --configurator certbot-external-auth:out \
        --certbot-external-auth:out-public-ip-logging-ok \
        -d "subdomain1.example.com" \
        -d "subdomain2.example.com" \
        --preferred-challenges dns \
        --certbot-external-auth:out-handler ./dns_check.py \
        --certbot-external-auth:out-dehydrated-dns \
        --logs-dir logs --config-dir conf --work-dir work \
        run

My error would be:

dns_check.py: error: unrecognized arguments: -M-H_CewSxuh-sXrrwHNN0cwZyefCddAk07OIAFgNGs g0XdZcrbFDS2ZawnxKHP5Z6jfD_giutI5ZFWhAb_IgU\n'

and to patch this issue locally I prepended a space to each argument in the argument list that begins with a -:

        # plugin.py line 722      
        # arg_list = [self._get_handler(), command] + list(args)

        #new                                                                                                                                
        safe_arglist = []
        for arg in list(args):
            if arg.startswith('-'):
                arg = arg.replace('-', ' -', 1)
            safe_arglist.append(arg)                                                                                  
        arg_list = [self._get_handler(), command] + safe_arglist

I'm happy to submit a PR though there may be a less hacky way to handle this.

Missing dependency "future"

After doing "pip install certbot-ext-auth", "certbot --version" errors out with "An unexpected error occurred: \ ModuleNotFoundError: No module named 'past'".
To get module past, amusingly I had to "pip install future".

It seems this is a forgotten dependency in pip.

TypeError: Object of type 'bytes' is not JSON serializable

Encountered exception during recovery
module 'types' has no attribute 'StringTypes'
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/certbot-0.15.0-py3.6.egg/certbot/auth_handler.py", line 115, in _solve_challenges
    resp = self.auth.perform(self.achalls)
  File "/usr/lib/python3.6/site-packages/certbot_external_auth-0.0.8-py3.6.egg/certbot_external_auth/plugin.py", line 204, in perform
    responses.append(mapping[achall.typ](achall))
  File "/usr/lib/python3.6/site-packages/certbot_external_auth-0.0.8-py3.6.egg/certbot_external_auth/plugin.py", line 477, in _perform_dns01_challenge
    self._json_out_and_wait(json_data)
  File "/usr/lib/python3.6/site-packages/certbot_external_auth-0.0.8-py3.6.egg/certbot_external_auth/plugin.py", line 781, in _json_out_and_wait
    self._json_out(data, True)
  File "/usr/lib/python3.6/site-packages/certbot_external_auth-0.0.8-py3.6.egg/certbot_external_auth/plugin.py", line 773, in _json_out
    json_str = json.dumps(data)
  File "/usr/lib/python3.6/json/__init__.py", line 231, in dumps
    return _default_encoder.encode(obj)
  File "/usr/lib/python3.6/json/encoder.py", line 199, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/usr/lib/python3.6/json/encoder.py", line 257, in iterencode
    return _iterencode(o, 0)
  File "/usr/lib/python3.6/json/encoder.py", line 180, in default
    o.__class__.__name__)
TypeError: Object of type 'bytes' is not JSON serializable

Most likely because how Python2 and Python3 differs?

certbot: error: unrecognized arguments:

I've been using certbot for years and it has never failed. Now it says unrecognized arguments.

certbot \
                --text \
                --agree-tos \
                --email $EMAIL \
                --configurator certbot-external-auth:out \
                --certbot-external-auth:out-public-ip-logging-ok \
                --preferred-challenges dns \
                --certbot-external-auth:out-handler /usr/local/bin/wc-ssl-dns-handler \
                --expand \
                certonly

And I now get:

usage: 
  certbot [SUBCOMMAND] [options] [-d DOMAIN] [-d DOMAIN] ...

Certbot can obtain and install HTTPS/TLS/SSL certificates.  By default,
it will attempt to use a webserver both for obtaining and installing the
certificate. 
certbot: error: unrecognized arguments: --certbot-external-auth:out-public-ip-logging-ok --certbot-external-auth:out-handler /usr/local/bin/wc-ssl-dns-handler

Plugin not found on CentOS 7 and 8

I have tried to install the plugin on CentOS 7. First, the encoding="utf-8" in setup.py must be removed, then the package could be installed with pip from the modified tarball. Certbot is version 1.7.0 which can be installed from Fedora EPEL for CentOS. I use this to renew HTTP-based certificates just fine. I then wanted to use certbot-external-auth to issue a wildcard cert with a DNS-01 challenge. CentOS 7 uses Python 2.7, as does the certbot package.

Running certbot plugins shows other plugins, but not the external-auth plugin. I have used strace -f -e trace=open certbot ... to see wihch Python modules are opened. All plugins open fine, also the external-auth plugin.py is opened. However, the plugin does not show up.

I suspected there might be problems with the old CentOS 7. I therefore also tried with CentOS 8 in a docker container docker run --rm -ti centos:8 /bin/bash. Inside the container I did:

yum install -y https://dl.fedoraproject.org/pub/epel/epel-release-latest-8.noarch.rpm
yum install -y certbot
pip-3 install certbot-ext-auth
certbot plugins

The output of the latter command shows standalone and webroot, but not external-auth.The debug log does not yield more info.

I have also tried installing certbot and certbot-ext-auth from pip within a CentOS 8 container (also requires to pip install future or an error that past could not be found occurs). This didn't work either, plugin still not shown.

Am I missing anything? Does the plugin need some adjustments for newer certbot versions?