Potential cert issue with cloud deployment

Technical Help
#1

While deploying to a cloud server, I noticed the following in the houston docker logs (see last 3 lines):

Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator standalone, Installer None
Registering without email!
Obtaining a new certificate
Performing the following challenges:
http-01 challenge for research.capitalmastery.app
Waiting for verification...
Cleaning up challenges
IMPORTANT NOTES:
 - Congratulations! Your certificate and chain have been saved at:
   /etc/letsencrypt/live/quantrocket.example.com/fullchain.pem
   Your key file has been saved at:
   /etc/letsencrypt/live/quantrocket.example.com/privkey.pem
   Your cert will expire on 2023-03-06. To obtain a new or tweaked
   version of this certificate in the future, simply run certbot
   again. To non-interactively renew *all* of your certificates, run
   "certbot renew"
 - Your account credentials have been saved in your Certbot
   configuration directory at /etc/letsencrypt. You should make a
   secure backup of this folder now. This configuration directory will
   also contain certificates and private keys obtained by Certbot so
   making regular backups of this folder is ideal.


An unexpected error occurred:
KeyError: 'PATH'
Please see the logfile '/tmp/tmpz1ot6tta' for more details.

I opened the referenced log file /tmp/tmpz1ot6tta and found the following inside:

2022-12-06 06:54:38,050:DEBUG:certbot.main:certbot version: 0.31.0
2022-12-06 06:54:38,051:DEBUG:certbot.main:Arguments: ['--deploy-hook', 'openresty -s reload']
2022-12-06 06:54:38,051:DEBUG:certbot.main:Discovered plugins: PluginsRegistry(PluginEntryPoint#manual,PluginEntryPoint#nginx,PluginEntryPoint#null,PluginEntryPoint#standalone,PluginEntryPoint#webroot)
2022-12-06 06:54:38,055:DEBUG:certbot.log:Exiting abnormally:
Traceback (most recent call last):
  File "/usr/bin/certbot", line 11, in <module>
    load_entry_point('certbot==0.31.0', 'console_scripts', 'certbot')()
  File "/usr/lib/python3/dist-packages/certbot/main.py", line 1342, in main
    args = cli.prepare_and_parse_args(plugins, cli_args)
  File "/usr/lib/python3/dist-packages/certbot/cli.py", line 1277, in prepare_and_parse_args
    return helpful.parse_args()
  File "/usr/lib/python3/dist-packages/certbot/cli.py", line 635, in parse_args
    hooks.validate_hooks(parsed_args)
  File "/usr/lib/python3/dist-packages/certbot/hooks.py", line 22, in validate_hooks
    validate_hook(config.deploy_hook, "deploy")
  File "/usr/lib/python3/dist-packages/certbot/hooks.py", line 49, in validate_hook
    if not _prog(cmd):
  File "/usr/lib/python3/dist-packages/certbot/hooks.py", line 35, in _prog
    if not util.exe_exists(shell_cmd):
  File "/usr/lib/python3/dist-packages/certbot/util.py", line 116, in exe_exists
    for path in os.environ["PATH"].split(os.pathsep):
  File "/usr/lib/python3.7/os.py", line 678, in __getitem__
    raise KeyError(key) from None
KeyError: 'PATH'
2022-12-06 06:54:38,055:ERROR:certbot.log:An unexpected error occurred:

Despite this error, the installation appears to have been successful. I'm able to connect to the Jupyter lab environment over https, and log in. Additional, I'm able to connect remotely from the quantrocket cli client.

In summary, is this something I need to worry about? If not, great; just want to make sure before getting too far down the road on this deployment.

Thanks in advance!

#2

It looks like the error occurred in the cert renewal command which runs periodically to see if the cert needs renewing. You could run the command manually to make sure it was just a transient error:

docker --context <context-name> compose exec houston certbot renew
#3

Hey @Brian, got the output below, which doesn't throw an error, but it doesn't look like it's actually trying to do a renew.

The only thing I can think of that may have caused issues is that I didn't set the HOUSTON_DOMIN, HOUSTON_USERNAME, and HOUSTON_PASSWORD environment variables; instead I set LETSENCRYPT_DOMAIN, BASIC_AUTH_USER, and BASIC_AUTH_PASSWD variables directly in my docker compose override file this cloud environment. Maybe the houston environment vars are being used for more than just a passthrough for basic auth/letsencrypt vars?

Output of certbot renew:

docker compose exec houston certbot renew
WARN[0000] The "HOUSTON_DOMAIN" variable is not set. Defaulting to a blank string. 
WARN[0000] The "HOUSTON_USERNAME" variable is not set. Defaulting to a blank string. 
WARN[0000] The "HOUSTON_PASSWORD" variable is not set. Defaulting to a blank string. 

Saving debug log to /var/log/letsencrypt/letsencrypt.log

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Processing /etc/letsencrypt/renewal/research.capitalmastery.app.conf
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Cert not yet due for renewal

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

The following certs are not due for renewal yet:
  /etc/letsencrypt/live/research.capitalmastery.app/fullchain.pem expires on 2023-03-06 (skipped)
No renewals were attempted.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
#4

That’s the output you want to see since the cert isn’t due for renewal, so I don’t think I would worry about anything.

#5

Thanks @Brian!

FYI I ended up forcing a renew and confirm everything is working as expected. For others, see command below.

# Select option 2 (standalone) when prompted for authentication method
docker compose exec houston certbot certonly --force-renew -d quantrocket.example.com

You can also do a dry run to make sure everything is working as below:

# Had to specify port as it wasn't being read from config as expected
docker compose exec houston certbot renew --dry-run --http-01-port 81