Comment by ZeroConcerns

Comment by ZeroConcerns a day ago

36 replies

I'm all for it -- it's hard to understate the extent to which LetsEncrypt has improved the WebPKI situation. Although the effective single-vendor situation isn't great, the "this is just something you only do via an automated API" approach is absolutely the right one. And certificate lifetimes measured in days work just fine with that.

The only things that continue to amaze me are the number of (mostly "enterprise") software products that simply won't get with the times (or get it wrong, like renewing the cert, but continuing to use the old one until something is manually restarted), and the countless IT departments that still don't support any kind of API for their internal domains...

crote a day ago

It's not single-vendor. The ACME protocol is also supported by the likes of GlobalSign, Sectigo, and Digicert.

You've got to remember that the reduction to a 45-day duration is industry-wide - driven by the browsers. Any CA not offering automated renewal (which in practice means ACME) is going to lose a lot of customers over the next few years.

  • ZeroConcerns a day ago

    Effectively single-vendor. I'm not aware of any ACME-compatible CAs that don't have pernicious limits on their free plans (and if there are, I'd love to hear!), and here in the EU we've even recently lost a rather big player...

    • arccy a day ago

      Google Trust Services: https://pki.goog/

      • rainsford 13 hours ago

        I'm glad there are free alternatives to Let's Encrypt, but a major PKI provider also being by far the largest browser provider feels like a disaster waiting to happen. The check on PKI providers doing the right thing is browsers including them (or not) in their trust stores. Having both sides of that equation being significantly controlled by the same entity fundamentally breaks the trust model of WebPKI. I'm sure Google has the best of intentions, but I don't see how that's in any way a workable approach to PKI security.

    • dmurray a day ago

      "multiple vendors, but only one of them is nice enough to give the product away for free" is not "effectively single-vendor".

      The other CAs aren't prohibitively priced for anyone who has a business need for lots of certificates, in case Let's Encrypt disappears or goes rogue.

      • ZeroConcerns a day ago

        > other CAs aren't prohibitively priced

        Not if you look at the per-cert pricing, but if you factor in the cost of "dealing with incompetent sales" and "convincing accounting to keep the contract going", they absolutely are.

    • arp242 a day ago

      Doesn't ZeroSSL do this? acme.sh has been using it as the default for the last few years. As I understand it, it basically offers the same as Let's Encrypt.

    • [removed] a day ago
      [deleted]
schmuckonwheels a day ago

> The only things that continue to amaze me are the number of (mostly "enterprise") software products that simply won't get with the times

Yeah, no one's rewriting a bunch of software to support automating a specific, internet-facing, sometimes-reliable CA.

Yes it's ACME, a standard you say. A standard protocol with nonstop changing profile requirements at LE's whim. Who's going to keep updating the software every 3 months to keep up? When the WebPKI sneeze in a different direction and change their minds yet again. Because 45 will become 30 will become 7 and they won't stop till the lifetime is 6 hours.

"Enterprise" products are more often than not using internal PKI so it's a waste.

I would like to see the metrics on how much time and resources are wasted babysitting all this automation vs. going in and updating a certificate manually once a year and not having to worry the automation will fail in a week.

  • homebrewer a day ago

    In circles I'm running in, automatic certificate renewal has not caused a single problem over 7 years of using it, and whatever time was spent on setting it up, has paid many times over, both in saving effort on renewal, and in putting out fires when (not if) someone forgets to renew a certificate. You just have to be careful picking your automation — I haven't been impressed with certbot, for example.

    Also, everything is using https now. Living in a low-income country, certificates were too expensive to use them where they weren't absolutely required, but not anymore. This is orthogonal to automation, I'm just pointing out that LE is not as demonic as you make it out to be.

    I'm afraid enterprise users are on their own, probably approximately no-one else is interested in going back to the old ways of doing it. (Maybe embedded.)

    • imtringued a day ago

      Forcing automation would be fine if the default software package (certbot) was any good but from my experience certbot is simply not fit for purpose. Certbot doesn't support the industry standard PKCS#12 format, which makes it extremely brittle for anyone using a Java based webserver. Instead it uses the non-standard PEM format which requires conversion before usage. That conversion step breaks all the time and requires manual intervention. It's ridiculous.

  • ZeroConcerns a day ago

    > I would like to see the metrics

    Well, I could regale you with my anecdotes on how all my 'grab a LetsEncrypt cert on Docker deploy and whenever remaining lifetime goes below 75%' services have not had a single TLS-related outage in at least a year, and how we just had a multi-day meltdown caused by a load-bearing cert that everyone forgot about expiring, but I doubt you'll be interested.

    I'm not here to take away your right to set up a few more 5-year+ internal CAs (or, hey, how about an entire self-signed hierarchy? can't be that hard...), and neither is LetsEncrypt. But on the Big Bad Internet (where Google is the leading cause of More Security, and LetsEncrypt is merely the leading vendor catering to that), things have moved on slightly.

  • rainsford 13 hours ago

    > I would like to see the metrics on how much time and resources are wasted babysitting all this automation vs. going in and updating a certificate manually once a year and not having to worry the automation will fail in a week.

    I would also like to see those metrics, because I strongly suspect the cost is dramatically in favor of automation, especially when you consider the improved security posture you get from short lived certificates. My personal experience with Let's Encrypt has been that the only hassle is in systems that don't implement automation, so I get the worst of both worlds having to effectively manually renew short lived certificates every 90 days.

    CRL based revocation is largely a failure, both because implementation is a problem and because knowing when to revoke a certificate is hard. So the only way to achieve the goal of security in WebPKI that's resilient to private key compromise is short-lived certificates. And the only way to implement those is with automated renewal.

  • crote a day ago

    The change in validity does not in any way alter the protocol itself. As mentioned in the linked blog post: if you've already got automated cert renewal, it'll almost certainly require zero change.

    After all, the logical approach is to schedule a daily cron job for your renewal process, which only contacts the server when the current cert has less than X days of validity remaining. Scheduling a one-off cron job every, say, 60 days would be extremely error-prone!

    With regards to future reductions: the point is to force companies to automate renewal, as they have proven to be unable to renew in time during incidents due to dozens of layers of bureaucracy. 45 days means one renewal a month, which is probably sufficient for that. Anything below 7 days isn't likely to happen, because at that point the industry considers certification revocation unnecessary.

    Internal PKI is not relevant here. Anyone with weird internal cert use shouldn't be using public certs anyways, so moving those people to certs backed by self-signed internal CAs is a win. They are free to use whatever validity they want with those.

    • basscomm 18 hours ago

      > the point is to force companies to automate renewal

      Cool. I'm a small-time webmaster with a couple of hobby sites with no more than a handful of visitors. Why do I need to set up automation to renew certs every 45 days, too?

      • wolrah 12 hours ago

        For the same reasons as forcing companies to do it.

        1. Revocation is a clusterfuck. Microsoft is currently failing to revoke tens of thousands of defective certificates for over seven months (the Baseline Requirements state that they should have been revoked within five days). Entrust was distrusted over repeated failures to revoke. Many TLS clients don't even bother to check revocation lists, or if they do they do it in a "fail-open" manner where inability to load the list does not prevent access which largely defeats the purpose. Short certificate lifetimes make revocation less important as both defective and compromised certificates age out rapidly, but without automation any meaningful reduction in lifetime directly translates to an increase in required effort which makes it a balancing act. With automation, however, reducing lifetimes is effectively free.

        2. Automation is mostly a one-time cost. Manual renewal is a repeating cost. I started using LE to add HTTPS to my personal site when it first became available to the public in 2016 and then started using it for my work systems when our GoDaddy certs came up for renewal a bit less than a year later. Since then out of roughly 50 systems pulling around 70 certs I've had to "babysit" two of them once each, both because they were using wildcard certs which I was a very early adopter of and something about how I had set them up was technically wrong but worked in the initial version. Compare this to the "old world" where every couple of years I had to generally engage vendor support on both our server platforms and our CA because it had been long enough that things had changed on both sides so doing things manually required learning new things. Mandating short lifetimes is good for everyone. This is part of why LE has used a short lifetime since the beginning, to strongly encourage people to not try to do it without automation.

        3. It's super easy. Lots of platforms have ACME support built in. For those that don't, packages like acme.sh are damn close to universal. If your system is able to expose a web server on port 80 it's basically trivial. If it's not, it's slightly harder. There's just not a good reason not to do it other than stubborn insistence. "Handcrafted TLS Certificates" just don't matter.

        • basscomm 9 hours ago

          I'm talking about managing two certificates so I can share a static site with a handful of friends. Each one takes about 10 minutes a year to update.

          Adding automation means I have to set up a process that I have to check up on at least once every 6.5 weeks to make sure it's still working.

      • Shish2k 13 hours ago

        I'm a small time webmaster and I haven't "set up" any automation - for my shared-hosting sites, the host has it built in; and for my self-hosted sites, the web server has it built in

        • iggldiggl 4 hours ago

          The problem is that this breaks down if you don't want to leak any obscure subdomains you might be using via CT-logs – shared hosting rarely supports DNS-based certificate renewals for wildcard certificates, and even less so for domains hosted by an external registrar.

          (Even for a fully self-hosted system you'd still have to figure out how to interface the certificate renewal mechanism with your DNS provider, so not as easy to set up as individual certificates for each subdomain.)

      • nickf 14 hours ago

        Is the certificate you use on your website any different to that on google.com? Does/could a browser know this and act differently?

  • happymellon a day ago

    > I would like to see the metrics on how much time and resources are wasted babysitting all this automation vs. going in and updating a certificate manually once a year and not having to worry the automation will fail in a week.

    I have multiple systems. Legacy that I've inherited, and modern that are able to automatically update their certs.

    The automated updates require almost zero maintenance, there was a week a couple of years ago when I had to switch some configuration for a new root cert.

    The legacy system requires manual installation and multiple weeks of my time every year because of the number of environments, and since generation of the certs requires me to file a request for someone to manually create it, they invariably typo something and it has to be redone everywhere.

    So multiple engineers, over multiple weeks?

    Manual process at a guess is £50k pa, while the automated is close to an annual £0?

  • patmorgan23 19 hours ago

    All the enterprise software needs to do is create an API for configuring the certificate in the product. Then they can integrate Certbot or one of the many other ACME solutions, or let the customer do it (they are enterprises after all).

  • ForHackernews a day ago

    I'll happily take failed automation with errors I can fix over being totally screwed when we find out the guy who did it every year left the company this year and nobody else knows anything about it.

  • arp242 a day ago

    > A standard protocol with nonstop changing profile requirements at LE's whim. Who's going to keep updating the software every 3 months to keep up?

    It really doesn't change that often. And whether this is a "breaking" change is something that's debatable (you shouldn't hard-code the cert lifetime, but I suspect many programs do, so de-facto it may or may not be a breaking change).

    If you look at the Go implementation for example (https://github.com/golang/crypto/commits/master/acme), then there haven't been any major changes since it was first written in 2018: just bugfixes and basic maintenance tasks. Maybe I missed a commit but it certainly doesn't require 3-month update cycles. You can probably take that 2018 code and it'll work fine today. And since it doesn't hard-code the 90 days it will continue to work in 2028 when this change is made. So it's more "you need to keep updating the software every decade to keep up".

riffic 18 hours ago

There's a slew of RFC documents that cover these related protocols so imagine that now means "requests for compliance".