I've been using Let's Encrypt with my Django websites for a few months running on my KnownHost VPS (which I absolutely love). It took a bit of work to get everything going but overall I have been happy with their free service and it certainly is saving me a lot of money on SSL certificates.
Initially I had issues installing the SSL certs. My initial errors were ones like:
WARN The domain “x.com” has failed domain control validation (The system queried for a temporary file at “”" target="_blank" class="externalLink ProxyLink" data-proxy-href="http://x.com/15136.BIN_AUTOSSL_CHECK_PL__.HD758vMuXhQNdjx5.tmp”" rel="nofollow">http://x.com/15136.BIN_AUTOSSL_CHECK_PL__.HD758vMuXhQNdjx5.tmp”, but the web server responded with the following error: 404 (NOT FOUND). A DNS or web server misconfiguration may exist.). at bin/autossl_check.pl line 449.
I was able to get things finally working by tweaking some virtual conf files with a couple Apache mod_rewrite rules and the help of cPanel's AutoSSL plugin.
Things were working fine until my 90 day free certificate expired and the AutoSSL plugin kept failing renewal of the Let's Encrypt certificate. This likely was from Django conflicts or some other configuration problems. At first cPanel AutoSSL was just not working and I couldn't get anything to help. So I went back to the drawing board and worked on the mod_rewrite rules until I was able to figure out ones that worked better.
If you are having issues getting your SSL certificate to renew or even initially install for Let's Encrypt then maybe these rules can help you. Before I show the rules (which are located a bit farther down in this post) I wanted to give an overview of Let's Encrypt and installing the cPanel AutoSSL plugin for those that might need help.
What Is Let's Encrypt
Let's Encrypt is a SSL certificate authority that offers free SSL certs (How It Works). This is in stark comparison to the vast majority of current certificate authorities that sell SSL/TLS certificates for a price based on the certificate type. Buying SSL certificates can get very costly. So using a valid free cert authority like Let's Encrypt is a great idea.
One key component to the free SSL certs that Let's Encrypt provides is that everything is built around automation. This is because all certificates they offer have a 90-day expiration. There is no exception and you cannot obtain a certificate with a longer expiration like you can get from other certificate authority companies. They also have limitations around the amount of certs you can install each day.
This really isn't too much of an issue because they provide the program (called certbot) to do it for you. Also there are other tools like cPanel's AutoSSL Plugin that can help as well.
Install Let's Encrypt SSL using cPanel AutoSSL Plugin
The cPanel AutoSSL plugin finally works with the free SSL service Let’s Encrypt on a cPanel/WHM service for versions above 58.0.17. You can see if your cPanel version is above 58.0.17 by logging into WHM checking the top right. If your version is older you may be able to use some things but maybe you should reach out to your hosting provider and look into a cPanel upgrade.
If you have the correct version then you are able to install the Let’s Encrypt script via the command line. It will also be available as a plugin from WHM version 60.
The command to run is:
/scripts/install_lets_encrypt_autossl_provider
Once complete, go into SSL/TSL on WHM and select Manage AutoSSL. Select the Let’s Encrypt SSL service and agree to any Terms of Service that might show up. Next you want to go to the Manage Users tab and determine which Domains you want to enable AutoSSL for.
To enable just select Enable AutoSSL for the domain and click the "Check" button. The cPanel plugin will then run through its steps in the next minute or two. To see if you are successful, go to the Logs tab, click refresh, and view the latest log. You should see something like this:
Log for the AutoSSL run for all users: Friday, March 31, 2017 2:54:02 AM GMT-0700 (Let’s Encrypt™)
2:54:02 AM This system has AutoSSL set to use “Let’s Encrypt™”.
2:54:04 AM Checking websites for “xxxxYourDomainxxxx” …
2:54:04 AM The website owned by “xxxxYourDomainxxxx” has a valid SSL certificate.
2:54:04 AM The system has completed the AutoSSL check for “xxxxYourDomainxxxx”.
2:54:06 AM The system has finished checking 1 users.
2:54:06 AM AutoSSL has deleted the expired log for the provider “LetsEncrypt” from Feb 28, 2017, 10:54:01 AM (UTC).
I suggest that you use a test domain to get everything working before you enable any main domain. I would do this because sometimes it can take a bit to ensure that you have everything configured correctly to get the SSL cert installed. I would also do your initial tests on a domain that does not have a SSL cert already.
Once you have things working I would run through 1 90 day cert cycle, then I would start enabling the other domains. Make sure that you put in a reminder to check your Let's Encrypt renewals for the next couple 90 day cycles just to make sure everything is working correctly.
A successful Let's Encrypt SSL renewal should have a log entry like this:
Log for the AutoSSL run for “xxxxYourDomainxxxx”: Thursday, April 6, 2017 10:38:31 AM GMT-0700 (Let’s Encrypt™)
10:38:31 AM This system has AutoSSL set to use “Let’s Encrypt™”.
10:38:31 AM Checking websites for “xxxxYourDomainxxxx” …
10:38:31 AM The website “xxxxYourDomainxxxx.com”, owned by “xxxxYourDomainxxxx”, has a faulty SSL certificate (AUTOSSL_READY_FOR_RENEWAL). AutoSSL will attempt to replace this certificate.
10:38:31 AM The system will attempt to renew SSL certificates for the following websites:
10:38:31 AM contactcs.com (xxxxYourDomainxxxx.com www.xxxxYourDomainxxxx.com mail.xxxxYourDomainxxxx.com)
10:38:37 AM SUCCESS The system has installed a new certificate onto “xxxxYourDomainxxxx”’s website “xxxxYourDomainxxxx.com”.
10:38:37 AM The system has completed the AutoSSL check for “xxxxYourDomainxxxx”.
You can get even more information on cPanel AutoSSL in their documentation on Managing AutoSSL.
Setting Up Apache Virtual Conf Files mod_rewrite Rules
When I first started setting up the mod_rewrite rules I opened a support ticket with KnownHost and cPanel. Because I was using Django as my web framework I guess it resulted in a bit of confusion and additional problems that you just don't see with most web hosting.
One of the primary issues with Django was that the URLs you access are defined very specifically and it isn't always easy to just allow some generic HTTP call to come in. Especially from the cPanel AutoSSL plugin. Added to that it seems that the AutoSSL plugin makes calls via HTTP primarily and that lead to some issues when I had the site setup to run on Django with HTTPS.
A lot of the suggestions I found on the support forums and other blogs was to redefine TMP files to a specific folder so we could bypass the restrictions. This was done via this mod_rewrite rule:
RewriteRule "^/(.*\.tmp)$" "/home/userid/public_html/$1"
Additionally, in some of my initial set up we determined that it made sense to call out the specific HTTP User Agent that the Let's Encrypt certbot was using in trying to set up the SSL certs. There were two different clients that we noticed and tried to call them in various ways:
RewriteCond %{HTTP_USER_AGENT} "^Cpanel-HTTP-Client\/1\.0$"
RewriteCond %{HTTP_USER_AGENT} "^COMODO\ DCV$"
and
RewriteCond %{HTTP_USER_AGENT} (COMODO\ DCV|Cpanel-HTTP-Client\/1\.0) [NC]
None of the above mod_rewrite rules worked for me in the long run and I do not suggest you use them. They all just led to the same type of 404 errors and the temp files that certbot and Let's Encrypt generated not being found. At one point, I was able to get things working and thought everything was going to be ok. It was 90 days later that the renewal of the SSL certificate failed that caused me to see that I had more work to do.
Ultimately, I went back to square one and generated a new virtual host configuration file with all new mod_rewrite rules based on the Visitor Access Logs from my domain. By doing this I was able to get the renewal to work. I even tried a new domain from scratch and the rules seemed to be fine as well.
Here is the virtual host configuration file that I came up with that is working for initial installs and renewals. This config file is for the STD HTTP vhost and not the SSL HTTPS one:
# Path: /usr/local/apache/conf/userdata/std/2_4/userid/domain.com/sslredirect.conf
RewriteEngine On
#####################
# cPanel AutoSSL code
#####################
RewriteCond %{HTTP_USER_AGENT} "^Cpanel-HTTP-Client(.*)" [NC]
RewriteRule "^\.BIN_AUTOSSL_CHECK_PL__\.(.*)" "/$1"
#####################
#####################
# Redirect all requests minus cPanel AutoSSL/Lets Encrypt to SSL site
#####################
RewriteCond %{HTTP_USER_AGENT} "!^Cpanel-HTTP-Client(.*)" [NC]
RewriteCond %{REQUEST_URI} "!^/\.well-known/acme-challenge/(.*)" [NC]
RewriteRule "^/(.*)" "https://www.yourdomain.com/$1" [R=301]
#####################
I noticed that for my client the primary HTTP_USER_AGENT I was seeing was: Cpanel-HTTP-Client/1.0. So I set up the rules to watch for that. But, just in case they change the version number I only look for the name and wildcard anything after it.
There was a request that came in from "Mozilla/5.0 (compatible; Let's Encrypt validation server; +https://www.letsencrypt.org)" after a successful registration. I assume it was to check that the cert installed, etc. But since it was a standard Mozilla HTTP_USER_AGENT (just like a normal user) I did not want to make any changes. The good news, though, was that it was checking the ".well-known/acme-challenge/" URL path so I could build around that which I will discuss later.
Next, I noticed that during initial installation the calls are made over HTTP and a "BIN_AUTOSSL_CHECK_PL" URL path seemed to be called and causing my initial 404 problems (see the error code at the top of this post). Because the URL they use had randomized characters I could not easily code it in Django for a redirect. This is where mod_rewrite came into the situation and I just take any call that has "BIN_AUTOSSL_CHECK_PL" in it and redirect it to the root path.
The big issue seemed to come from renewals of the SSL cert. This happened because after you get HTTPS going you make sure to put redirects on your website to take any HTTP traffic and resend it to HTTPS. From what I can tell, the Let's Encrypt process uses HTTP for the URL it gives you. If you don't have the right code in your SSL vhost file, then the renewal will fail.
I didn't want duplicate cPanel Auto SSL mod_rewrite rules in both my virtual host config files (for the Apache STD and SSL conf directories) so I decided to handle things a bit differently. Basically, I put in a rule that if the HTTP request came from the "Cpanel-HTTP-Client" HTTP_USER_AGENT or was looking for the ".well-known/acme-challenge/" REQUEST_URI path, that it should not be redirected to HTTPs. All other traffic did get sent to HTTPS though.
This seemed to do the trick and now I can have a simple set of rules in my STD HTTP vhost conf file and nothing in the SSL HTTPS vhost conf file related to Let's Encrypt or cPanel AutoSSL. This is very helpful to me because my SSL virtual configuration file is already complicated enough with my Django configuration settings since I use mod_wsgi to diplay the Django site, etc.
Troubleshooting cPanel AutoSSL Let's Encrypt Certificate Renewal Failures
So you have Let's Encrypt working on the cPanel AutoSSL plugin, but it tries to renew your certificate and fails. Or maybe the initial AutoSSL SSL cert install fails. What do you do?
To trouble shoot AutoSSL issues use the visitor logs from the domain you are working with because it gives you the best understanding of what is happening with the requests as they come in. Here is an example from a recent cert renewal request:
# Visitor Log : HTTP
IP: xxx.xxx.xxx.xxx
URL: /.well-known/acme-challenge/SZ1QVCYPYN4W7HVGTVT94RX865U3SMV8
Status: 301
User Agent: Cpanel-HTTP-Client/1.0
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# Visitor Log : HTTPS
IP: xxx.xxx.xxx.xxx
URL: /.well-known/acme-challenge/SZ1QVCYPYN4W7HVGTVT94RX865U3SMV8
Status: error 404
User Agent: Cpanel-HTTP-Client/1.0
I then use the error log from the AutoSSL plugin to give additional detail. As an example, I noticed that certbot was looking for a specific URL but it was coming in as HTTP which was being 301 redirected to HTTPS. The HTTPS request would fail of course because Django wasn't set up to understand it and there were no virtual config file mod_rewrite rules in place either.
If you also look you can see that the specific user agent was Cpanel-HTTP-Client and the URL was .well-known/acme-challenge/. Instead of making generic large scale wildcard calls in my vhost file, I decided to make it very specific in definition.
By making changes, restarting Apache, and rechecking the domain in the AutoSSL plugin, you should be able to get the exact configuration needed to make the renewal work. Then, in 90 days check again to see if the settings worked. If so, you should be fine in applying the settings to the other domains you use.
Here is a valid visitor log after the certificate was installed and everything was working. You should be getting 200 response codes from the web server. There will be no requests to HTTPS, only HTTP logs because you are handling the initial HTTP requests correctly. Notice that there are two User Agent types.
# Visitor Log : HTTP
IP: xxx.xxx.xxx.xxx
URL: /.well-known/acme-challenge/0wv5NWS0uROhscyrg0C6lSJ2cK5uicDxNa-Xn_c8xmk
Status: success 200
User Agent: Mozilla/5.0 (compatible; Let's Encrypt validation server; +https://www.letsencrypt.org)
IP: xxx.xxx.xxx.xxx
URL: /.well-known/acme-challenge/0wv5NWS0uROhscyrg0C6lSJ2cK5uicDxNa-Xn_c8xmk
Status: success 200
User Agent: Cpanel-HTTP-Client/1.0
Conclusion
I hope the mod_rewrite virtual configuration above help solve your Let's Encrypt issues. You probably can adjust it and put it into a .htaccess file as well if need be. Either way, the key thing to remember it identifying the User Agent and path it is requesting. Then building rules around handling it.
If you need anything please reach out to me or let me know what other Django related problems you are having!