I recently had to renew the SSL certificate for my web server, nothing exciting about that but Certbot created a new directory for the key because I had removed some domains (moved to a different web server). This normally isn’t a big deal, change the Apache configuration to the new file names and run the “reload” command. My monitoring system initially said that the SSL certificate wasn’t going to expire in the near future so it looked fine. Then an hour later my monitoring system told me that the certificate was about to expire, apparently the old certificate came back!
I viewed my site with my web browser and the new certificate was being used, it seemed strange. Then I did more tests with gnutls-cli which revealed that exactly half the connections got the new certificate and half got the old one. Because my web server isn’t doing anything particularly demanding the mpm_event configuration only starts 2 servers, and even that may be excessive for what it does. So it seems that the Apache reload command had reloaded the configuration on one mpm_event server but not the other!
Fortunately this was something that was easy to test and was something that was automatically tested. If the change that didn’t get accepted was something small it would be a particularly insidious bug.
I haven’t yet tried to reproduce this. But if I get the time I’ll do so and file a bug report.
I’ve already noticed old certificates being served after a reload and came to the habit of restarting apache2 when changing the certificate or the SSL/TLS configuration.
AFAIK apache starts new (so called) generation of processes with new config and tells old generation to shut down when they finish handling clients (in case of graceful restart). I had cases where old generation didn’t die fully … but cannot tell if master process was directing any traffic to old generation clients or just existing clients where hanging there.
niol: Thanks, it’s good to know that others are seeing it.
a: The simple way to test is to run a program like gnutls-cli that has one TCP connection per process run, that proved that the old Apache process was taking new connections.