Standing on our own two feet(letsencrypt.org)
letsencrypt.org
Standing on our own two feet
https://letsencrypt.org/2020/11/06/own-two-feet.html
199 comments
My professor was one of the founders of Let's Encrypt. He said it was basically a game theory problem. Whichever CA defects gets money. The rest get nothing. IdenTrust decided to get that money, because they thought that if they didn't, a different CA would.
And that's why only a competitive market drives down prices.
IdenTrust doesn't care about random websites, they care about HIPPA, enterprise, government, securing documents and emails, etc.
Exactly, but the funny thing is that Chrome and Firefox (Desktop at least) are no longer showing the differentiating green lock for those high-end (EV & OV) certs, but the same neutral-looking lock as those Domain-Validated (DV) certs that Let's Encrypt is issuing.
I'm very grateful for IdenTrust for having made that move. I just hope it won't hurt their business too much because of that.
I'm very grateful for IdenTrust for having made that move. I just hope it won't hurt their business too much because of that.
I've never understood why browsers didn't show the SSL Common Name or other agreed upon identifier, in place of a little lock. Why do I have to click 4 times in Firefox Linux Desktop, just to see info on the cert?
So this is perhaps why there is no EV or OV differentiation. Who cares? Of what use is an EV cert, if no one even checks the name. Or further, knows if the bank (for example) uses that CA?
I think in such a context, 'green' and 'no-green' is just non-helpful to validate anything. Sadly, 1 person out of 1000? actually care about encryption, or even know what SSL is. Maybe only 1 out of 10000 know about EV.
Sometimes I just become sad, when I think of the lack of general knowledge about fairly important things.
So this is perhaps why there is no EV or OV differentiation. Who cares? Of what use is an EV cert, if no one even checks the name. Or further, knows if the bank (for example) uses that CA?
I think in such a context, 'green' and 'no-green' is just non-helpful to validate anything. Sadly, 1 person out of 1000? actually care about encryption, or even know what SSL is. Maybe only 1 out of 10000 know about EV.
Sometimes I just become sad, when I think of the lack of general knowledge about fairly important things.
For PKIX (and thus in your web browser) leaf certificates the X.509 Common Name is only permitted to be textually equivalent to one of the SANs (Subject Alternative Names, the Internet's way to write a name for a machine) in the certificate. So that's either a dnsName or an ipAddress. This is grandfathered in because it's how Netscape worked last century before PKIX was standardised and thus before SANs existed to do this properly.
So it would be prohibited to issue leaf certificates with a CN that's a human meaningful name like "Google" or "Hacker News" because that violates PKIX.
It doesn't matter anyway, the only enforcement that really matters for HTTPS is the mechanical enforcement by the user agent, because there are way too many HTTPS transactions for the human to realistically assess the certificate shown for each transaction and decide if it's OK.
So it would be prohibited to issue leaf certificates with a CN that's a human meaningful name like "Google" or "Hacker News" because that violates PKIX.
It doesn't matter anyway, the only enforcement that really matters for HTTPS is the mechanical enforcement by the user agent, because there are way too many HTTPS transactions for the human to realistically assess the certificate shown for each transaction and decide if it's OK.
How would you propose verifying that agreed upon identifier?
Validating human-readable names, be that of individuals or corporations, would be opening a can of worms. Domain validation is already decidedly non-trivial.
Validating human-readable names, be that of individuals or corporations, would be opening a can of worms. Domain validation is already decidedly non-trivial.
Agreed, I don't think we'll ever see this because most people don't care. I'd guess greater than 95% of people, really 99% of people, couldn't tell you the difference between HTTP and HTTPS.
It just should work for them, and the browser should enforce it. I think the tech world is biased to think consumers are more technically inclined due to the people they are around. I do not work in computer tech. No-one I work with, all of whom have some form of an engineering degree unrelated to computers, could tell you the difference or care less.
It just should work for them, and the browser should enforce it. I think the tech world is biased to think consumers are more technically inclined due to the people they are around. I do not work in computer tech. No-one I work with, all of whom have some form of an engineering degree unrelated to computers, could tell you the difference or care less.
Doesn't really matter if inertia is keeping them using Identrust. They're not going to go to the trouble of switching providers just to save a few bucks. If that was their concern there were much cheaper options available even before Let's Encrypt
As a consumer, I think it's regrettable that they don't show EV in a different way. It served for me as a signal that the website were less likely to be scammers.
But maybe Mozilla & Google, were aware of it being used like that and thought that EV certs were not reliable enough to be used as a signal of trustworthiness?
But maybe Mozilla & Google, were aware of it being used like that and thought that EV certs were not reliable enough to be used as a signal of trustworthiness?
Does anyone in enterprise actually need publicly trusted certificates for documents and email? Seems like it's an inside-the-firewall Exchange server for internal traffic, and a white-label "secure messaging center" portal for external traffic.
IdenTrust's buisness also spans to managing private CAs for companies, which includes managing the HSM and private keys. Also, the companies who hire IdenTrust and similar companies are not that involved in technology. Also, security experts who can manage this safely is a tad harder to find and requests higher wages than your standard IT staff.
TLDR: yes, but some companies wants another company to manage their certs.
TLDR: yes, but some companies wants another company to manage their certs.
In those cases IdenTrust bought an insane amount of goodwill from some of the most technical people online by supporting Let’s Encrypt.
They also helped basically make it a requirement by allowing google to flag http pages as insecure and convince the public that it's necessary
Using a public CA is far better for security than a custom private one. It's a pain having to install the certificate on every client, server, piece of software, etc. and in my experience this inevitably leads to people disabling certificate checking as part of troubleshooting and this being left on. Also, sometimes people need to access documents and emails from home computers and the company may use some devices on which it isn't possible to install the CA
So exposing your internal infrastructure to the whole world and risking a 3rd party (CA) turning the keys (literally) to your kingdom to someone else is better than someone making a mistake that’s very easy to discover and correct?
> Also, sometimes people need to access documents and emails from home computers and the company may use some devices on which it isn't possible to install the CA
That’s a plus as far most security professionals are concerned
> Also, sometimes people need to access documents and emails from home computers and the company may use some devices on which it isn't possible to install the CA
That’s a plus as far most security professionals are concerned
> So exposing your internal infrastructure to the whole world and risking a 3rd party (CA) turning the keys (literally) to your kingdom to someone else is better than someone making a mistake that’s very easy to discover and correct?
That's not how certificates work. The CA doesn't have your private key. They could theoretically sign a fake certificate with your hostname but that risk is still present if you use a private CA and is mitigated by certificate transparency
That's not how certificates work. The CA doesn't have your private key. They could theoretically sign a fake certificate with your hostname but that risk is still present if you use a private CA and is mitigated by certificate transparency
Yes, should’ve been more clear on that they sign a cert without your knowledge and hand to to someone performing mitm. How is that risk present when you roll your own PKI and validate against your private CA (or intermediate) only?
Regarding CT I’m not aware of any clients other than browsers actually enforcing that.
Regarding CT I’m not aware of any clients other than browsers actually enforcing that.
Typically an internal CA adds to the certificate trust store rather than replacing it.
Yes you are correct here (although I’ve seen both methods). At least 3rd party won’t easily know which hostnames to fake though
It seems like a case of “commoditize your complement” to me. https://www.gwern.net/Complement
They offer a variety of enterprises services and can now capture more of the marginal consumer value relative to competitors still fighting for the cert issuing market.
They offer a variety of enterprises services and can now capture more of the marginal consumer value relative to competitors still fighting for the cert issuing market.
Always better to be doing the disruption rather be the one being disrupted.
Perhaps they saw the writing on the walls and wanted to boost their standing in the post LetsEncrypt world.
Perhaps they saw the writing on the walls and wanted to boost their standing in the post LetsEncrypt world.
Yeah they likely got a stash of money for it while the other CAs got nothing. First mover advantage I guess. It's not that they'd have been able to prevent it anyways, as the root stores are not under their control, especially when Let's encrypt is being supported, even started, by the entities which run the root stores (Mozilla, through Firefox, and Google, through Android). Ultimately it's the entities which run and deploy the root stores who have ultimate say on which CA gets to issue certs and which doesn't.
Maybe IdenTrust will now offer an ACME compatible endpoint and offer signed, paid certs with their CA. Or another CA will.
I wonder whether IdenTrust imagined that a five year cross signed root ca would be too little a timespan to get wide adoption.
Btw... Wouldn't it be possible to just add a new root ca to android? Maybe an app could simplify delivery?
I wonder whether IdenTrust imagined that a five year cross signed root ca would be too little a timespan to get wide adoption.
Btw... Wouldn't it be possible to just add a new root ca to android? Maybe an app could simplify delivery?
> Maybe an app could simplify delivery?
I'd be very surprised if an app without root privileges could install a new root certificate. If an app installed a malicious (or even just a poor quality) certificate, that would be a pretty big compromise to the OS.
What is strange to me though, is that it seems like the OS should have a mechanism to update the root certs independently of the OS itself. Then again, not updating root certs is a way to put an expiration date on a phone, forcing customers to buy more phones...
I'd be very surprised if an app without root privileges could install a new root certificate. If an app installed a malicious (or even just a poor quality) certificate, that would be a pretty big compromise to the OS.
What is strange to me though, is that it seems like the OS should have a mechanism to update the root certs independently of the OS itself. Then again, not updating root certs is a way to put an expiration date on a phone, forcing customers to buy more phones...
I would imagine that the app could make the delivery smoother than "download a file on the filesystem, look for a menu somewhere where to add the root ca".
Maybe a single confirmation box "would you like to add this ca" would work.
Maybe a single confirmation box "would you like to add this ca" would work.
> I'd be very surprised if an app without root privileges could install a new root certificate.
Its not like the OS can actually withstand the app though, looking at a years out-of-date OS with thousands of accumulated known bugs.
Its not like the OS can actually withstand the app though, looking at a years out-of-date OS with thousands of accumulated known bugs.
Android has always had an "install certificate from SD card" option, so it's absolutely possible, just very annoying
Starting with (I think) Android 6, this will be accompanied with a non-discardable "Somebody might be tracking you!!!" scare warning in the menu bar. I'm not a fan of the idea of educating users to ignore something like that.
The article says firefox app comes with its own up to date certificates which they maintain outside of the os, so there's that solution apparently.
I think a fairly manageable path forward is underway which makes this a smaller issue than it first seems:
(1) Firefox already uses its own root store
(2) App developers can include additional roots in addition to the system root store: https://developer.android.com/training/articles/security-con...
(3) Chrome is migrating to using it's own store: "Historically, Chrome has integrated with the Root Store provided by the platform on which it is running. Chrome is in the process of transitioning certificate verification to use a common implementation on all platforms where it's under application control, namely Android, Chrome OS, Linux, Windows, and macOS. Apple policies prevent the Chrome Root Store and verifier from being used on Chrome for iOS."
https://www.chromium.org/Home/chromium-security/root-ca-poli...
(1) Firefox already uses its own root store
(2) App developers can include additional roots in addition to the system root store: https://developer.android.com/training/articles/security-con...
(3) Chrome is migrating to using it's own store: "Historically, Chrome has integrated with the Root Store provided by the platform on which it is running. Chrome is in the process of transitioning certificate verification to use a common implementation on all platforms where it's under application control, namely Android, Chrome OS, Linux, Windows, and macOS. Apple policies prevent the Chrome Root Store and verifier from being used on Chrome for iOS."
https://www.chromium.org/Home/chromium-security/root-ca-poli...
Let’s Encrypt cross-signature with IdenTrust "DST Root X3" is ending on September 1, 2021 but 33.8% of Android devices are running versions under 7.1 which don't trust Let’s Encrypt new root certificate "ISRG Root X1"
Workaround is Firefox Mobile (because it ships with its own root certs), but that's a significant burden to place on the user.
Also the post says that Firefox doesn't work on Androids older than 5.0 which according to the dashboard are still 5.9% of devices.
For those older devices, the only option is to install the new root certificate.
Anyways, there are billions of Android devices out there. 33% of those is a large number. You can't just tell all of them that they are wrong.
If this happens, people will move away from Let's encrypt in masses. They don't realize yet how self-harming this really is.
For those older devices, the only option is to install the new root certificate.
Anyways, there are billions of Android devices out there. 33% of those is a large number. You can't just tell all of them that they are wrong.
If this happens, people will move away from Let's encrypt in masses. They don't realize yet how self-harming this really is.
Root certificate updates are a massive security issue. Blaming Let's Encrypt is blaming one of the canaries for the coal mine disaster. 33% of Android devices don't and can't get up to date root certificates is an impressive security crisis that grows worse by the year (look at the other root expirations and the crazy workarounds that for instance Netflix has been doing to still work on older Android devices). Shouldn't the blame squarely be on Google, the Android OEMs, and the phone carriers for allowing this disaster to happen in the first place?
I realize that is a tough message to get out to users and site owners are going to be in the cross-fire, but it seems better to try to work for solidarity in pointing fingers at the right direction and the right direction certainly isn't Let's Encrypt.
I realize that is a tough message to get out to users and site owners are going to be in the cross-fire, but it seems better to try to work for solidarity in pointing fingers at the right direction and the right direction certainly isn't Let's Encrypt.
That makes me curious, what workarounds did Netflix employ?
It was the BBC I was actually thinking about, and it's a part of this article (which also mentions this Let's Encrypt root change):
https://scotthelme.co.uk/impending-doom-root-ca-expiring-leg...
Previous HN discussion on that article: https://news.ycombinator.com/item?id=23455463
https://scotthelme.co.uk/impending-doom-root-ca-expiring-leg...
Previous HN discussion on that article: https://news.ycombinator.com/item?id=23455463
When you control the client, it's simple, you can do pretty much anything: embed your own HTTP stack, TLS stack, your QUIC stack, or simply your PKI, or subset of the webPKI.
Yes the issue is really severe of most deployed Android devices not getting security updates, either at all, or the devices are used well beyond the update period.
But this is not up to Let's Encrypt to solve. They market themselves to build products for the mass market instead of small niches of the market, say, everyone who buys a new phone every year. But then they also have to treat their product like a mass market product, and if Android users still use older versions of the OS, then Let's Encrypt should adopt for that.
But this is not up to Let's Encrypt to solve. They market themselves to build products for the mass market instead of small niches of the market, say, everyone who buys a new phone every year. But then they also have to treat their product like a mass market product, and if Android users still use older versions of the OS, then Let's Encrypt should adopt for that.
This problem isn't unique to Let's Encrypt. Let's Encrypt is not the only entity with certificates signed by DST Root X3. And as these devices get older, more root certificates will expire. What happens when all of them expire?
What is unique about Let's Encrypt, is they may have a harder time getting cross-signed by a CA that will still have a valid root cert on these devices for a significant amount of time, because, as has been pointed out in other comments, Let's Encrypt is disrupting the CA industry.
What is unique about Let's Encrypt, is they may have a harder time getting cross-signed by a CA that will still have a valid root cert on these devices for a significant amount of time, because, as has been pointed out in other comments, Let's Encrypt is disrupting the CA industry.
They will be happy to accept that blame, as long as you fork over your $ for a new device. Planned obsolescence can have many components, this is just one of them.
If you're running Android 5.0, you also haven't benefited from updates that remove CAs that have since been shown to be untrustworthy. I think that's far worse than being unable to visit sites that use LetsEncrypt certificates.
It was really short-sighted of Google to make the system cert bundle something that can't be updated without a full OS update. There should be an OTA mechanism that allows it to be updated through the Play Store or through some other means that isn't reliant upon lazy device manufacturers.
It was really short-sighted of Google to make the system cert bundle something that can't be updated without a full OS update. There should be an OTA mechanism that allows it to be updated through the Play Store or through some other means that isn't reliant upon lazy device manufacturers.
To be fair they haven't gotten mant other security updates either. It's just a bad situation all around.
33% of devices, but per the article, only 1-5% of traffic on sites using LetsEncrypt. I don't see any site owners moving to a different CA when this affects less than 5% of their visitors, who are likely the poorest fraction of their userbase , i.e. probably not many paying users.
How many other root certificates are going to be expiring in the next 3 years that older Android won't have updates for as well? Probably more than 1.
on androids older than 5, the browser is the old android browser instead of chrome. how many sites out there still test compatability with that?
even without having to click through security warnings, the web is horribly broken on old android devices. the overlap of sites using letsencrypt and sites that care about people using android <5 has got to be vanishingly small. this isn't going to cause a move away from letsencrypt.
even without having to click through security warnings, the web is horribly broken on old android devices. the overlap of sites using letsencrypt and sites that care about people using android <5 has got to be vanishingly small. this isn't going to cause a move away from letsencrypt.
Not sure about that. Move away from Let's Encrypt to what? More likely, most smaller to medium sized sites will say forget those old Android guys.
Too cheap to update can be seen as too cheap to buy your product or service, so I can see this happening.
I found at least Buypass offering a gratis ACME product "Buypass Go SSL". They have roots which are deployed at least since Android 4.1, which covers way more Android devices (according to the Android Studio statistics, >99%):
https://android.googlesource.com/platform/libcore/+/android-...
I'm not sure whether that particular root is being used for their Go SSL product. If so, Buypass might be a good alternative to migrate to.
From what I read though, they do require an E-Mail address, so you've got to keep that in mind.
https://android.googlesource.com/platform/libcore/+/android-...
I'm not sure whether that particular root is being used for their Go SSL product. If so, Buypass might be a good alternative to migrate to.
From what I read though, they do require an E-Mail address, so you've got to keep that in mind.
> Also the post says that Firefox doesn't work on Androids older than 5.0 which according to the dashboard are still 5.9% of devices.
For those older devices, the only option is to install the new root certificate.
Microsoft Edge still gets updates on Android 4.4 KitKat
Microsoft Edge still gets updates on Android 4.4 KitKat
I don't think Microsft Edge embeds its own root store on Android.
Is there enough incentive for Microsoft to add a root store to Edge by next September? How hard is it to make that addition?
I may be wrong, but the effort to switch to your own root store is more doing it securely, than the difficulty of switching from system frameworks to your own SSL/HTTP transport layers. So to put another way, straight forward to do mediocre job, not as trivial to do a good or great job.
Root store and TLS/HTTP library are separate concerns. You can use the system root store with your own libraries, or you can use your own root store with the system libraries.
On an Android 4.4 device, you should probably skip the system root store and the system libraries, and if you're already doing it for those phones, you might as well do it for all the phones.
On an Android 4.4 device, you should probably skip the system root store and the system libraries, and if you're already doing it for those phones, you might as well do it for all the phones.
> Root store and TLS/HTTP library are separate concerns.
In the context of this thread (aka older Android devices), they aren’t truly separate concerns. You really need to do both. My point is that doing both is relatively straightforward, but doing the root store part is fairly easy to do it in a mediocre way and be brittle / insecure.
In the context of this thread (aka older Android devices), they aren’t truly separate concerns. You really need to do both. My point is that doing both is relatively straightforward, but doing the root store part is fairly easy to do it in a mediocre way and be brittle / insecure.
In the post, they advertise that they are going to continue to offer a service that provides certificates signed with the old one.
... which will work until September 2021.
That's not gonna help as the root will expire, so the devices stuck with the older root will still shows errors when trying to use them.
You can still install the last version of Firefox to support Android 4.x.
I don’t think they have a choice. Reading between the lines, the CA who cross-signed their previous root doesn’t really want to continue doing so (or asked for a lot more money) because LE usage reached levels that are just too risky. I don’t blame them: a single bad actor found doing something particularly nefarious with a LE certificate might lose them the trust their business is literally built upon. I don’t see any other CA queueing up to help what is typically their commercial nemesis. And as they say, at some point they would have to do it anyway, might as well rip the plaster off.
Agreed, but what could someone do with a domain validation cert (as issued by LE) that would be so nefarious to damage IdenTrust's reputation?
Unfortunately, on my Moto G5 Plus (which is not an high-end device), I've found that Firefox on Android is slower than Chrome (specifically the Bromite fork). I think that Firefox may not be a good solution for low-spec or older phones running older Android versions.
Firefox is not a workaround for non-web mobile apps. Lots of them will stop working unless they do cert pinning / have their own CA bundle or simply stop using LE..
A Workaround is not something that the user does.
The most impressive thing it that Let’s Encrypt are the ones who are trying to fix a problem that should be fixed by phone manufactures and telcos.
I can see why, but I would also have like the phones to “break” so the owners would avoid those brands in the future, and pick one who care enough to push out update.
Still, I can blame Let’s Encrypt, they just want to be the good guys, and the do it so beautifully and transparently.
I can see why, but I would also have like the phones to “break” so the owners would avoid those brands in the future, and pick one who care enough to push out update.
Still, I can blame Let’s Encrypt, they just want to be the good guys, and the do it so beautifully and transparently.
It probably wouldn't encourage many people to upgrade their phones, though. They would probably just see the website as broken and either click through the warning or abandon it completely
Nearly 60% of the web is using Let's Encrypt certs now. It's not just the one of two sites are going to break, it's going to be half the web.
Good. Those devices are unsafe, and should not be used.
I think you might be ignorant with regards to how the rest of the world uses the internet.
But for certain businesses, there is a case to be made for not serving these kinds of customers. Someone using a shitty old Android phone might not even be able/willing to pay for your product, and if they do, might cost more in technical support and/or fraud (due to their device being vulnerable) than what they bring in revenue.
When you are a bank or a crypto exchange, that might make sense. When you are the municipality of a village in rural India or the company that handles all vehicle registrations, it does not.
You cannot gatekeep with such sweeping statements. People have old phones for lots of reasons. Others will have to serve those people for lots of other reasons.
You cannot gatekeep with such sweeping statements. People have old phones for lots of reasons. Others will have to serve those people for lots of other reasons.
"https almost everywhere".
Thanks G!
Thanks G!
Somewhat related, but in other thread two weeks ago people complain Google has too much power over Android ecosystem: https://news.ycombinator.com/item?id=24917918
Now, here people are suggesting Google should somehow update the old Androids.
Be damned one way or the other.
Now, here people are suggesting Google should somehow update the old Androids.
Be damned one way or the other.
Yes and Yes.
And both are reasonable and not excluding.
Google sells you a pocket computer with a locked down OS, not for your safety but to control the ability to run ads.
If they cared about user security, they would provide updates, no matter how "slow" (their excuse) the device gets.
If they didn't want full control to show ads (ads are downloaded by the GooglePlayServices, which is pretty much the kernel of all your android experience) then it would be trivial to install other android distributions like replicant.
hence, both a reasonable and google is evil. They want full control and do not care about (your) security updates.
And both are reasonable and not excluding.
Google sells you a pocket computer with a locked down OS, not for your safety but to control the ability to run ads.
If they cared about user security, they would provide updates, no matter how "slow" (their excuse) the device gets.
If they didn't want full control to show ads (ads are downloaded by the GooglePlayServices, which is pretty much the kernel of all your android experience) then it would be trivial to install other android distributions like replicant.
hence, both a reasonable and google is evil. They want full control and do not care about (your) security updates.
Google does provide updates. It's the device manufacturers and/or mobile operators who choose not to push them.
> Google does provide updates
That is a lie and you know it. Google updates the OS, but they are also directly responsible for many products they sold themselves with their brand. And those have as much updates as any other company, well maybe one or two more.
I happen to have two devices bought directly from google. One is stuck on android 2.3 and another on 4.0, both full of security holes, not updated because "it would be too slow" when in reality i can't even install replicant et al because google never worked with the component providers to offer compatible binary blobs for the hardware.
Yeah, android itself is opensource (mostly because it is built on top of GPLed linux code so they do not have an option) but 99% of what makes your phone run is a proprietary binary-only code provided by the likes of Qualcomm etc. And why phone manufacturers, google included, use the options with closed source binary blobs? To save $5 or so from the BOM cost in production.
That is a lie and you know it. Google updates the OS, but they are also directly responsible for many products they sold themselves with their brand. And those have as much updates as any other company, well maybe one or two more.
I happen to have two devices bought directly from google. One is stuck on android 2.3 and another on 4.0, both full of security holes, not updated because "it would be too slow" when in reality i can't even install replicant et al because google never worked with the component providers to offer compatible binary blobs for the hardware.
Yeah, android itself is opensource (mostly because it is built on top of GPLed linux code so they do not have an option) but 99% of what makes your phone run is a proprietary binary-only code provided by the likes of Qualcomm etc. And why phone manufacturers, google included, use the options with closed source binary blobs? To save $5 or so from the BOM cost in production.
But that model is flawed, and it's been more than a decade to learn that.
The interesting thing is: We already have a model that works much better, and it's been around for longer. If you buy a computer with Windows it is completely normal that you still get your updates from Microsoft, even if your computer is built by a company that may no longer exist by the time you install the update.
(That's not to say Windows and Microsoft don't have their own security issues - but the idea that "we provide an OS and we add a middle man for OS updates that by all experience doesn't do his job" is a good model is at this point preposterous.)
Users of the Android OS are not Google's customers.
Most of them are. :) Except for Huawei, I haven't seen an Android user not using Google Play Services. There is probably statistically-insignificant number of users just having F-Droid on a LineageOS who do not use Play Store, Push services or don't consume their ads. So most Android users are indirectly also Google users.
I'd wager that's untrue. Anyone who buys an app, or in app purchase, is directly a Google customer.
I'd also argue that even if someone uses Play Store services, but doesn't pay a penny, they're still a "customer" in a looser sense of the word. They're engaging in the market-place, likely using free apps subsidized by advertising. Even if you want to argue that they're the product instead of the client (and I'd be inclined to agree), they're still revenue-generating users.
I'd also argue that even if someone uses Play Store services, but doesn't pay a penny, they're still a "customer" in a looser sense of the word. They're engaging in the market-place, likely using free apps subsidized by advertising. Even if you want to argue that they're the product instead of the client (and I'd be inclined to agree), they're still revenue-generating users.
People complaining is pretty constant, yeah. People even complain about complaining.
True,
But we need Google to step up because we know that the manufacturers and carriers won't.
But we need Google to step up because we know that the manufacturers and carriers won't.
Could Google possibly be able (before were discuss willingness) to push an update to root certificate via Play Services?
I'd like to think that anyone not using Play Services (i.e. Android with no Play) is likely using a custom browser, and would heed a call to switch to Firefox.
The problem with some devices in Africa would be that many people will using older phone often don't have enough data for the big Play updates to succeed.
Teens often buy 10-100MB of data so they can use WhatsApp. (If you're from Southern Africa, and disagree with this, hit me up, you probably need to spend some time in a village ;) )
I'd like to think that anyone not using Play Services (i.e. Android with no Play) is likely using a custom browser, and would heed a call to switch to Firefox.
The problem with some devices in Africa would be that many people will using older phone often don't have enough data for the big Play updates to succeed.
Teens often buy 10-100MB of data so they can use WhatsApp. (If you're from Southern Africa, and disagree with this, hit me up, you probably need to spend some time in a village ;) )
No play services on an Android phone in the US probably implies willingness to tinker. No play services on an Android phone in China only implies it's an Android phone. In the developing world, it most likely implies a very low cost Android phone of Chinese origin.
Bundling things that need timely updates with the OS with no mechanism to update them individually is a design error. Things like root certificates, time zone databases, leap second information, and even TLS libraries need to be updated on a regular basis. These items should be distributed outside of the general upgrade process, even if the general upgrade process worked (which is clearly not the case). Alternatively, root certs and TLS libraries could be bundled with applications as needed. You could probably have a stable core x.509 library and cipher algorithms bundled with the OS, so that the application level TLS library can be kept small. You still need to get tzdb updates out though.
In an ideal world, large OS vendors could work with carriers to get this small set of updates zero-rated in exchange for making sure they are very small and background downloaded only at times of low network congestion.
Bundling things that need timely updates with the OS with no mechanism to update them individually is a design error. Things like root certificates, time zone databases, leap second information, and even TLS libraries need to be updated on a regular basis. These items should be distributed outside of the general upgrade process, even if the general upgrade process worked (which is clearly not the case). Alternatively, root certs and TLS libraries could be bundled with applications as needed. You could probably have a stable core x.509 library and cipher algorithms bundled with the OS, so that the application level TLS library can be kept small. You still need to get tzdb updates out though.
In an ideal world, large OS vendors could work with carriers to get this small set of updates zero-rated in exchange for making sure they are very small and background downloaded only at times of low network congestion.
In an ideal world carriers wouldn't have a say in what software updates were installed on my phone. Comcast doesn't control the software on the computers it services. Why should Telus control what updates are made available for my phone?
Because they're the ones who push updates over the cell network. Comcast absolutely controls what software you run on your modem. You can update "out of band" manually, at least on recent Android Pixel phones. Any other manufacturer could also make their updates public, but since installing the one not for your carrier band makes the phone unusable as a phone, it's not likely to be common.
Comcast has 0 control over what runs on my modem (Spectrum in my case). As long as the modem is DOCSIS compliant, it will work.
The same applies to unlocked phones. The service provider has 0 control over what I am running on that phone, and they don't control the updates (the OEM does), but as long as the baseband firmware complies with established standards, the phone will work. This was mandated by law some years back in the US and I am certain it's been the case in the EU for longer.
What you seem to be referring to is telco customized phones (subsidized ones), and in those cases you'd be correct.
The same applies to unlocked phones. The service provider has 0 control over what I am running on that phone, and they don't control the updates (the OEM does), but as long as the baseband firmware complies with established standards, the phone will work. This was mandated by law some years back in the US and I am certain it's been the case in the EU for longer.
What you seem to be referring to is telco customized phones (subsidized ones), and in those cases you'd be correct.
Because security and privacy is the compromise Google made for dominance: Letting OEMs and carriers do what they want is what sold them on Android.
I'd absolutely agree that this is a design error though: We'll be better off when Android is dead and gone as a platform.
I'd absolutely agree that this is a design error though: We'll be better off when Android is dead and gone as a platform.
Can't you buy hardware directly and then just put the carrier's SIM card?
Starting with Android 10, Google can push updates to lots of system components (media codecs, android frameworks, tzdata, …) https://android-developers.googleblog.com/2019/05/fresher-os...
But those on older versions are screwed. It's really a major fuck-up that Google didn't do this for root certs since the beginning of Android.
But those on older versions are screwed. It's really a major fuck-up that Google didn't do this for root certs since the beginning of Android.
Actually I was surprised that there would be such an easy fix : Switching to Firefox. Which is apparently around 70MB, apparently affordable from what you wrote and definitely worth it if it allows you to unlock a chunk of the internet. So no need for an improbable and costly Play update.
That won't fix any other apps though will it? Anything that uses chrome webview for example.
Oh, I did not think about that. On Android 8, you make firefox the default ... (renderer ?) for other apps but idk if it's the case for older versions of Android.
On the other hand, as others have said, these devices are generally quite painful to browse on so accessing the wweb version of many apps could be a solution, plus firefox will let you place shortcuts on your home screen . I also wonder how LineageOS works on those old devices. Could be another solution.
I know little about conditions in African villages. Do they ever make city trips where they have access to free wifi with unlimited data?
How much does that much data typically cost??
Here are some South Africa prices: https://www.mtn.co.za/recharge/data
Valid for a day: 25MB for $0.32; Valid for a week: 50MB for $0.64; Valid for a month: 100MB for $1.28, 1GB for $6.35
Valid for a day: 25MB for $0.32; Valid for a week: 50MB for $0.64; Valid for a month: 100MB for $1.28, 1GB for $6.35
> The remaining 33.8% of Android devices will eventually start getting certificate errors when users visit sites that have a Let’s Encrypt certificate. In our communications with large integrators, we have found that this represents around 1-5% of traffic to their sites.
This one-third of Android devices only yields 5% of traffic? Interesting.
This one-third of Android devices only yields 5% of traffic? Interesting.
I have a 2010 Android smartphone and it's painfully slow to navigate the modern web, almost unbearable. Browsing news websites is simply not worth my time of waiting for the phone to download and process 22MB of JS, CSS, and graphics. Being on wifi makes no difference; it's the CPU choking to render all that cruft.
So yeah, 5% of traffic makes sense.
So yeah, 5% of traffic makes sense.
In firefox (at least in desktop version) you can disable javascript and css. Not sure if they are still downloaded.
For scripts: if script is disabled for a document, scripts are not downloaded. See https://searchfox.org/mozilla-central/rev/a5d9abfda1e26b1207... as of today
For stylesheets, I'm not certain how you're disabling them. Depending on how you do it, they may or may not get downloaded. The most common ways of disabling them result in them not being downloaded.
For stylesheets, I'm not certain how you're disabling them. Depending on how you do it, they may or may not get downloaded. The most common ways of disabling them result in them not being downloaded.
View - Page style - No style
That one does not prevent downloading; just prevents application.
there's a lot of old android phones out there that don't get used for anything other than making phone calls or sending texts.
This also depends on the country. For instance in Poland android has 99% market share. So 1/3 Poland residents will experience issues.
Crappy expensive internet means it's not instinct to squeeze a quick browsing session into every free minute
More interesting that % of traffic in this case would be unique users. But it's not surprising to me that people using Androids on older devices are using them to access the internet less; they're generally not as nice to use, but also if they were heavily used, they may have worn out by now.
This might also concern you if your app is running on older android devices, and you have a backend that uses letsencrypt certificates, while the app is relying on the system root CAs.
A fix for this could be to add Let's Encrypt's root CA to your app.
A fix for this could be to add Let's Encrypt's root CA to your app.
> As of January 11, 2021, we’re planning to make a change to our API so that ACME clients will, by default, serve a certificate chain that leads to ISRG Root X1.
Ouch. So anyone who wants to support older devices 3 months from now needs to add `--preferred-chain "DST Root CA X3"` to their certbot command. When that chain is completely retired in September next year, certbot appears to fallback on the default (even with that argument present).
Ouch. So anyone who wants to support older devices 3 months from now needs to add `--preferred-chain "DST Root CA X3"` to their certbot command. When that chain is completely retired in September next year, certbot appears to fallback on the default (even with that argument present).
It appears the built-in time expiry period in the trust relationships between various organisations and clients will increasing become a weak point in the world in the future. Whether these moving parts are in trust only for three months, three years or more is not the issue, it is the fact that periodic resync and retrust needs to happen, means that it may result in a cascadable failure of our infrastructure in the future. Say if a nuclear disaster / earthquake take down one city, will it lead to paralysis of servers and services across the world? Perhaps it is time to start auditing the Internet and identify the weak points.
That doesn’t feel like a weak point to me. In the case you described, trust likely is broken. As a user, I may not care and instruct my browser to ignore the fact that the certificate is broken because I understand the likely cause.
I find the marketing spin in the title and the article unfortunate (standing on our own feet). The simple truth is they've decided getting another cross-signature not worth their trouble so now us users must expect some breakages the scale of which doesn't appear to be well understood.
Also, while the article focuses on the Android, my first thought was about all those outdated CentOS/Debian/etc boxes/VMs/containers that silently curl something from a cron job or such and that will all of a sudden stop working. So I expect a lot of infrastructure breakages.
Also, while the article focuses on the Android, my first thought was about all those outdated CentOS/Debian/etc boxes/VMs/containers that silently curl something from a cron job or such and that will all of a sudden stop working. So I expect a lot of infrastructure breakages.
The company I work at is in a high-growth phase and we are going to be expanding our global audience this coming year through various channels (SEO, performance marketing, sales, etc.). A 1-5% hit in potential customer traffic is not going to fly. Is my only option here to get off LetsEncrypt? A bit of a vent, but I would 100% had paid for a version of LetsEncrypt that supported their costs for the x-signature with IdenTrust, although I know and respect that would be off-brand for LE.
You certainly can't go to IdenTrust now either. No matter what you do, you'll lose access to those old-phone-people eventually, and paying would only lengthen the time slightly. Just like you already lost access to people with even older android phones. Some of my family is still on androids as old as version 2.
Are you on LetsEncrypt currently? From my experience working on legacy enterprise, I'd say to stay on there. Add the flag and you will get a few more months out of it '--preferred-chain "DST Root CA X3"'
Android 6 is 2015. root and intermediates CA have a 10 and 5 year lifespan. I am afraid you might not be able to find something that work on old phones and new phones.
Even if you do find an older CA vendor that has an ancient CA and is willing to sign (you will be forced into an enterprise contract that will take months to negotiate), it's going to be retired anytime soon and break everywhere.
Last but not least. Old phones are stuck on old versions of SSL/TLS, they're not able to connect to recent websites irrelevant of the certificates. Your site is probably no exception and cut the old protocols a long time ago.
Android 6 is 2015. root and intermediates CA have a 10 and 5 year lifespan. I am afraid you might not be able to find something that work on old phones and new phones.
Even if you do find an older CA vendor that has an ancient CA and is willing to sign (you will be forced into an enterprise contract that will take months to negotiate), it's going to be retired anytime soon and break everywhere.
Last but not least. Old phones are stuck on old versions of SSL/TLS, they're not able to connect to recent websites irrelevant of the certificates. Your site is probably no exception and cut the old protocols a long time ago.
Is there any way you can measure how many of your users are on an unsupported Android version?
I think most phones include it in their user agent string.
For some reason, Let's Encrypt failed to mention that older versions of Android can just import the ISRG Root X1 certificate, which solves the problem for all apps.
https://www.stoutner.com/lets-encrypt-isrg-root-x1-and-priva...
https://www.stoutner.com/lets-encrypt-isrg-root-x1-and-priva...
Someone could plausibly write a high-quality exploit for Android 7 that installs the ISRG root CA certificate.
/me runs
/me runs
They propose to install Firefox to work around the root certificate problem on old android devices. But can’t you just manually install their root certificate on most phones?
It probably looks a lot less sketchy to your users to tell them to install a browser they've probably heard of and may have used in the past than install a root certificate. Plus, that doesn't fix the problem of other certificates expiring, only extends it.
>Plus, that doesn't fix the problem of other certificates expiring, only extends it.
Although I agree fully on the sketchiness part, installing the root is a fix.
ISRG Root X1 expires in 2035. Not one of these problematic Android devices will be online anymore.
Although I agree fully on the sketchiness part, installing the root is a fix.
ISRG Root X1 expires in 2035. Not one of these problematic Android devices will be online anymore.
But to have the whole modern web work you'll need to install more than one root.
I'm not sure if it's the same on older versions, but on recent Android versions, that requires a rooted device.
Interestingly, it appears to be back in 11. You are right it wasn't possible for some set of versions, not sure how far you have to go back for it to be possible again.
They appear to have added it back with a big warning screen similar to what they do for VPNs and stuff telling users it could compromise them, which is reasonable. It was a pain you couldn't before.
They appear to have added it back with a big warning screen similar to what they do for VPNs and stuff telling users it could compromise them, which is reasonable. It was a pain you couldn't before.
I see you've guessed why I'm on Android 7! The updater refuses to work if you have su.apk in your system directory. Can't simply ignore that file now, can we?
I too have wondered why IdenTrust would want to do this. As has been mentioned in this thread, it appears they focus on enterprise-level customers, governments, medical, among others.
Generally speaking, the way that new competitors enter a given market is with low-cost options that are often inferior to established players. Then, as those entrants expand upmarket by offering better and improved products, the existing/established players abandon parts of their downmarket products to the new entrants. This cycle repeats until there's nowhere left for the established players to go. At that point, these upstarts can often replace the existing players and become the dominant ones.
I'm not sure if the above was a deliberate strategic move on the part of IdenTrust or not, but in cross-signing the Let's Encrypt certificate, it effectively killed off the potential for new players in the low-cost TLS/SSL certificate market because there's no margin in $0. [Citation needed] Further, because the purpose of Let's Encrypt is to serve the base level of the market [1] with no apparent desire (as per the parent organization which is effectively a non-profit/public-benefit organization) to expand upmarket. This move would appear to solidify (whether intentional or not) the position of larger players who cater to larger customers while keeping any potential newer players from disrupting the space.
Aside: I love Let's Encrypt and have about a dozen or so certificates issued through them that I am in charge of. They're awesome and kudos to their team for what they've been able to accomplish. When they first offered certificates, the 90-day validity period felt very restrictive. Now it feels great because the certificates are automatically rotated every 60 days per various automation tools and painful certificate renewals are very much a thing of the past for me.
[1] https://letsencrypt.org/about/
Generally speaking, the way that new competitors enter a given market is with low-cost options that are often inferior to established players. Then, as those entrants expand upmarket by offering better and improved products, the existing/established players abandon parts of their downmarket products to the new entrants. This cycle repeats until there's nowhere left for the established players to go. At that point, these upstarts can often replace the existing players and become the dominant ones.
I'm not sure if the above was a deliberate strategic move on the part of IdenTrust or not, but in cross-signing the Let's Encrypt certificate, it effectively killed off the potential for new players in the low-cost TLS/SSL certificate market because there's no margin in $0. [Citation needed] Further, because the purpose of Let's Encrypt is to serve the base level of the market [1] with no apparent desire (as per the parent organization which is effectively a non-profit/public-benefit organization) to expand upmarket. This move would appear to solidify (whether intentional or not) the position of larger players who cater to larger customers while keeping any potential newer players from disrupting the space.
Aside: I love Let's Encrypt and have about a dozen or so certificates issued through them that I am in charge of. They're awesome and kudos to their team for what they've been able to accomplish. When they first offered certificates, the 90-day validity period felt very restrictive. Now it feels great because the certificates are automatically rotated every 60 days per various automation tools and painful certificate renewals are very much a thing of the past for me.
[1] https://letsencrypt.org/about/
i hate how google puts warnings on non-ssl sites. why doe a static page that has no forms need ssl? non-ssl worked fine for 20 years for webpages and google comes along and says noooo not good enough.
Recently there was a browser zero-day observed in the wild that operated by MITM'ing HTTP connections and injecting the payload into the response. You're thinking of HTTPS as protecting what information you send, but it also protects what you receive; with an HTTP connection, anyone in the middle can make your browser receive anything they want.
When things have gotten so bad that all you need to do to own a browser is to connect to it the last desperate measure is to try to keep all the badness away.
The NSA can see what pages you read, and men in the middle can modify the page to insert malicious JS or ads or whatever without HTTPS.
Ok - next question then: why do browsers block self-signed certs? If Lets Encrypt now allows any domain to get a cert, what's the harm in a self-signed cert? Seems like a step up from plain HTTP.
What you are trusting, when you trust a CA, is that it will only issue a certificate for a domain to someone it has verified as the owner. A self-signed cert could be from a man in the middle attacker.
An active MITM is a relatively more exotic threat than a passive observer. In the days when certificates cost money, there was a legitimate argument that you shouldn't need the whole elaborate active-MITM defense just to get protection against passive snooping. But now that you can get both for free... just use Let's Encrypt.
An active MITM is a relatively more exotic threat than a passive observer. In the days when certificates cost money, there was a legitimate argument that you shouldn't need the whole elaborate active-MITM defense just to get protection against passive snooping. But now that you can get both for free... just use Let's Encrypt.
Small nit, verification is only meant to prove control over the domain, not ownership.
The theory here is a self-signed cert could be from anyone (including the NSA) and you wouldn't know. Unless you explicitly trusted the certificate you were using, like enterprises do.
NSA can probably already issue certs from some of the widely-trusted roots. If someone as big as NSA wanted to MITM you, you wouldn't notice...
I feel like this is an oversight due to the path we took to arrive here. HTTP website: no warning, "insecure" mark in the URL bar. Trusted HTTPS website: no warning, "secure" mark in the URL bar. Untrusted HTTPS website: impossible to open in modern browsers by normal users. Really, a non-expired, otherwise-valid self-signed certificate should be treated the same as an HTTP website is. But since LetsEncrypt made self-signed certificates much less common, I doubt there's going to be energy put into improving the experience with them.
As well as the affirmatively "insecure" marking in the URL bar for unencrypted HTTP, it's not a Secure Context (except localhost) and so modern browsers don't offer any new features for this case.
If you're writing an HTML 4.0 web page that's no issue. If you were building a video chat, or you were going to add geolocation to your "Where's my closest store?" page or you want to send Notifications, or you've got a framework that needs Service Workers, those APIs are Secure Context only and won't exist.
The general rule of thumb is, if you could polyfill it, then it's available even without Secure Context, otherwise you need Secure Context to get the new feature. Some older features are being retrospectively given this treatment.
If you're writing an HTML 4.0 web page that's no issue. If you were building a video chat, or you were going to add geolocation to your "Where's my closest store?" page or you want to send Notifications, or you've got a framework that needs Service Workers, those APIs are Secure Context only and won't exist.
The general rule of thumb is, if you could polyfill it, then it's available even without Secure Context, otherwise you need Secure Context to get the new feature. Some older features are being retrospectively given this treatment.
An MITM attack on an HTTPS connection looks exactly like an untrusted HTTPS connection, and the main purpose of HTTPS is to prevent a MITM. The browser should rightly scream louder here than it does for an unsecure HTTP connection.
The man-in-the-middle can self-sign their own certs and present it to you as the site's own self-signed cert. Unless you have some way to verify self-signed certs out-of-band, they're useless.
They’re not useless. They stop passive adversaries, like the Australian government. They just don’t stop active adversaries.
Browser security warnings imply https > http > self signed https. The correct order of should be https > self signed https > http.
Browser security warnings imply https > http > self signed https. The correct order of should be https > self signed https > http.
We're getting there, HTTP is going to be marked as insecure in the future as well. It's just the massive amount of HTTP sites that couldn't get marked as insecure before, due to the then resulting warning fatigue in users.
Allowing untrusted certificates for https, and showing just a warning for them, would make it impossible for websites to ensure that their traffic is not intercepted
Interesting point. BTW I love your blog and your work on ShareJS :)
[deleted]
To be fair, they probably can anyways, since the site willingly handed over their keys to Cloudflare, or rely on some Google CDN, host on EC2, etc.
It’s the second or third tier nation state actors that you can really make a difference with.
It’s the second or third tier nation state actors that you can really make a difference with.
The NSA can still see that just via the metadata of me making the connections necessary to load the encrypted page contents.
If they are focused on you then they can probably do that although it may take considerable work, such as spidering an entire web site to correlate resources on the site against timing and size information.
Without encryption we know the NSA takes the approach of just bulk collecting everything they can to filter after the fact.
So rather than an assigned agent trying to discern what Public Enemy #1 Lammy is up to right now, and painstakingly concluding you have probably just read a Wikipedia article about Jaffa cakes, if you don't have encryption what happens is that the automation notes "Lammy looked at http://en.wikipedia.org/wiki/Jaffa_Cakes" as you click and it goes into the vast heap of stuff snooped that second even though you aren't under any suspicion and it's probably irrelevant.
This bulk surveillance is an unjustified attack on everybody's privacy.
Without encryption we know the NSA takes the approach of just bulk collecting everything they can to filter after the fact.
So rather than an assigned agent trying to discern what Public Enemy #1 Lammy is up to right now, and painstakingly concluding you have probably just read a Wikipedia article about Jaffa cakes, if you don't have encryption what happens is that the automation notes "Lammy looked at http://en.wikipedia.org/wiki/Jaffa_Cakes" as you click and it goes into the vast heap of stuff snooped that second even though you aren't under any suspicion and it's probably irrelevant.
This bulk surveillance is an unjustified attack on everybody's privacy.
As a site owner that is somehow not my problem. It's not my fault that you have a shitty ISP. I can understand and support that argument.
I do TLS only for my stuff however.
I do TLS only for my stuff however.
> It's not my fault that you have a shitty ISP.
Really don't think you're grasping this and lack an understanding of packet routing.
Please type:
Apache will happily serve you an insecure connection, they are the 80th top site on the internet. This type of stuff should be called out by technologists and it's incredibly disappointing when its handwaved away.
Really don't think you're grasping this and lack an understanding of packet routing.
Please type:
traceroute apache.org
Apache will happily serve you an insecure connection, they are the 80th top site on the internet. This type of stuff should be called out by technologists and it's incredibly disappointing when its handwaved away.
That is still not my problem as a site owner.
Story time! Couple years ago I worked in a company in Asia. My boss went to China for work and bought some network equipment for the office: routers, access points etc. We didn't really need them, it's just so cheap he wanted to see how well it works. After setting them up and got it to work, we continue to use our devices as usual. But when we visit a company internal tool/dashboard page that we built ourselves, an ad banners starts to showed up on the page! We were baffled, is our server compromised? Are the computers we're using caught some malware? But it's happening on all of our devices, computers, phones, tablet. Then we start to suspect it's the new equipment that is injecting the ads! Following this, we also noticed it doesn't modify HTTPS sites and that was the last clue of the puzzle. We pulled all the new equipment and sworn to never trust anything from China that has a price tag that's too good to be true.
ISP injecting scripts into unencrypted pages are pretty common, but a router injecting ads to all unencrypted traffics is a new low.
A MITM can add a "Donate" button to news.ycombinator.com. The money doesn't actually go to Hacker News, though. It goes to the MITM.
A lot of shared hosts support LetsEncrypt now. I actually installed them on mine and to my surprise I saw a massive reduction in spam attacks also.
That seems reasonable. LE is doing good work & hardly their fault Android is a fragmented mess.
Does anyone have experiences with ZeroSSL? Caddy has been building in support so I think it could be a drop-in replacement for Caddy/CertMagic/ACMEx users.
ACMEz* ;)
Seconding regecks' comment.
We're gradually making ZeroSSL a default CA for Caddy.
(I am currently implementing multi-CA support into Caddy and CertMagic, so that Caddy will be able to use both Let's Encrypt and ZeroSSL for redundancy. It's the first server to support this!)
This is a good thing for the ecosystem.
Seconding regecks' comment.
We're gradually making ZeroSSL a default CA for Caddy.
(I am currently implementing multi-CA support into Caddy and CertMagic, so that Caddy will be able to use both Let's Encrypt and ZeroSSL for redundancy. It's the first server to support this!)
This is a good thing for the ecosystem.
> We're gradually making ZeroSSL a default CA for Caddy.
As in, replacing LE as the default, or supplementing it? (And if the former, why?)
As in, replacing LE as the default, or supplementing it? (And if the former, why?)
Note how I mentioned that Caddy will be the first server to support redundant ACME CAs, so we'll use both ZeroSSL, and Let's Encrypt for redundancy.
Hi! Could I ask a somewhat unrelated question about using Let's Encrypt with Caddy? I've been trying to help some folks (in education) get wildcard subdomain certificates to work on their Google Cloud machines via lego_deprecated's purported gcloud support in Caddy v2 (we've tried to follow the instructions and all), but we've been running into issues and it's been incredibly frustrating to figure out how to resolve them. I recall one of the errors we got was "No TXT record found at _acme-challenge.subdomain.domain.tld", but it was hard to see all of them because most of the errors we'd see would be rate-limit errors. Things were so much easier and everything worked in Caddy v1, but ever since we upgraded to v2, we have no idea how to make it work with gcloud (the instructions haven't gotten it working for us), and there seems to be a lack of any working examples on the internet. Do you know if anyone has had success with gcloud at all? Would you have any guidance on how to proceed? Currently they're running on expired certificates and we have no idea how to renew them via Caddy, and it's not clear to me how to even do it out-of-band either.
To avoid Let's Encrypt rate limits, please use the staging endpoint, as documented:
- https://letsencrypt.org/docs/staging-environment/ - https://caddyserver.com/docs/automatic-https#testing
For more help, please ask on our forums! I don't use Google Cloud but it is more likely that somebody there does: https://caddy.community -- otherwise, time to roll up your sleeves and get to work, forge the answer for others, I suppose!
- https://letsencrypt.org/docs/staging-environment/ - https://caddyserver.com/docs/automatic-https#testing
For more help, please ask on our forums! I don't use Google Cloud but it is more likely that somebody there does: https://caddy.community -- otherwise, time to roll up your sleeves and get to work, forge the answer for others, I suppose!
Okay thanks!
Just tried to switch to them from LE since I don't want to just drop 33% of Android users, but it looks like their ACME implementation is not RFC8555-compliant as their 'newAccount' endpoint can only be used only once in violation of section 7.3.1 of the RFC. They also seem to be pushing people to use their own proprietary API instead. So thanks but no thanks.
ZeroSSL is operated by Sectigo (Comodo). It works fine with Certbot and should work fine with any ACME client.
I did run into some random 503s occasionally, but that was pretty soon after it was launched. Maybe just a few teething issues.
I did run into some random 503s occasionally, but that was pretty soon after it was launched. Maybe just a few teething issues.
The answer there would be strong "rights to repair" legislation. In the most affected markets it would be a viable option to go to the phone shop and have a new operating system image installed for 20 Euros/Dollars.
But those markets don't have the legal power against Google or Samsung.
And markets that would have the legal power, don't care about unnecessary electronic waste ruining the planet.
(Sent from Android 4.1 without Playstore.)
But those markets don't have the legal power against Google or Samsung.
And markets that would have the legal power, don't care about unnecessary electronic waste ruining the planet.
(Sent from Android 4.1 without Playstore.)
The scenario you describe unfortunately requires more than just right to repair. It requires the SoC vendors to either keep updating their kernels, or to open source their driver blobs. Without these, you can't build a proper new OS image, you're stuck on the latest one provided by the SoC manufacturer.
In practice, updates might be possible, but you'll often see hardware stopping working.
If it is "just" bluetooth, that may be fine. When networking (wifi/GSM) or the screen stops working, it probably is not.
I've done a fair bit of upgrades of ancient android phones to CyanogenMod, LineageOs and even an accidental Ubuntu Touch. Fairly common that things like the camera or bluetooth stop working (partly), but for many owners of old phones, that is certainly a trade to make. "I never use Bluetooth, what could you use it for?", "Oh, but then I'll just use this jack-cable for my sonos").
What I'm trying to say: yes: updates require SoC vendors to help, or at least stay out of the way. But no, that does not mean one cannot ever update at all.
If it is "just" bluetooth, that may be fine. When networking (wifi/GSM) or the screen stops working, it probably is not.
I've done a fair bit of upgrades of ancient android phones to CyanogenMod, LineageOs and even an accidental Ubuntu Touch. Fairly common that things like the camera or bluetooth stop working (partly), but for many owners of old phones, that is certainly a trade to make. "I never use Bluetooth, what could you use it for?", "Oh, but then I'll just use this jack-cable for my sonos").
What I'm trying to say: yes: updates require SoC vendors to help, or at least stay out of the way. But no, that does not mean one cannot ever update at all.
Does IdenTrust have another root cert that has a later expiration date, that is also included in the trust store of OSes farther back than 2016? If so, why can't LetsEncrypt ask them to start cross-signing with a different root cert?
(Obviously IdenTrust is under no obligation to do so, but since they've done this much, it's not a stretch to hope they'd do more, even if they want to charge for it.)
(Obviously IdenTrust is under no obligation to do so, but since they've done this much, it's not a stretch to hope they'd do more, even if they want to charge for it.)
IdenTrust owns two newer roots which are widely trusted today, IdenTrust Commercial Root CA 1 and IdenTrust Public Sector Root CA 1 but they were only created in 2014.
Perhaps somebody has a list of what's in the trust store for various historical Android builds, I do not.
But I think you can assume that the team at Let's Encrypt have considered any options that might work and decided that either they wouldn't make enough difference to be worthwhile or have asked and been told it isn't possible or would be too expensive to make sense.
Perhaps somebody has a list of what's in the trust store for various historical Android builds, I do not.
But I think you can assume that the team at Let's Encrypt have considered any options that might work and decided that either they wouldn't make enough difference to be worthwhile or have asked and been told it isn't possible or would be too expensive to make sense.
My vote is for trusting Let's Encrypt, because SSL protects my users (and myself, and each other) from MITM attacks! However, it is interesting to ask what "trust" means in this context, since I don't know the people behind Let's Encrypt, so I don't trust them in that way. What are we trusting the organizations associated with those certs to do, or not to do?
Good. Let’s Encrypt is giving a massive headstart. They’re also giving an easy escape hatch for those who can’t get things ready in time.
Good judgment.
Good judgment.
Question: If your site certificate is signed by a chain of intermediate certificates leading to a root that your browser knows, when you browser checks all the signatures at time T does it require that all the certificates be valid at time T, or does it just require that each certificate was valid at the time it was used for signing?
now the corporates got a valid point, why you dont want to use lets encrypt?
still the 33% of the devices is a quite a large number to consider.
still the 33% of the devices is a quite a large number to consider.
But like they said in the article, those 33% of Android phones represent "1-5% of the traffic" of the "large integrators" websites that LE communicated with.
Well that's an easy choice, lose 1-5% of traffic or pay $100 for a certificate from a vendor whose root doesn't expire next year?
1-5% of traffic that comes from people using devices that are at least 4 years old. Someone who can't or won't upgrade from a phone that still uses Android Marshmallow is probably not bringing in much revenue.
Exactly. The “easy choice” is to drop support for those users. I suspect most web developers won’t even notice.
It’s a pity - there’s no essential reason why old phones with new batteries should get worse over time. But software kills them in so many ways.
It’s a pity - there’s no essential reason why old phones with new batteries should get worse over time. But software kills them in so many ways.
As long as this statement is true. They also dont's say who these big integrators are and the situation may be widely different between integrators and fields.
Now that they’re standing on their own feet can they also offer S/MIME certificates for email encryption?
Are there no companies out there with very old root signatures that could be acquired? It seems to me like those sigs are valuable IP just like well-known domains (like example.com) - I wonder how much one is worth.
What about iOS? No word about it in this article.
According to apple [1] ISRG Root X1 is available all the way down to iOS 10.
So that would be every iPhone since the 5.
As far as marketshare goes iOS 13 and 12 make up 94% of devices [2]. So I am guessing its an insignificant amount of actual users.
[1] https://support.apple.com/en-us/HT204132
[2] https://developer.apple.com/support/app-store/
As far as marketshare goes iOS 13 and 12 make up 94% of devices [2]. So I am guessing its an insignificant amount of actual users.
[1] https://support.apple.com/en-us/HT204132
[2] https://developer.apple.com/support/app-store/
Or macOS, or Windows, or the BSDs. Would be nice to see a table of which versions will have issues or not.
At least for Windows: if browser support is being considered, Firefox is the last browser that somewhat works (both IE and Chtome fails because of TLS 1.2) and it includes the ISRG root (because duh), and if an application still updates on XP there is a good chance that it uses OpenSSL or derivatives (which uses a different root set, likely Mozilla's).
Natively? Windows 7, assuming you have installed all updates before January. Microsoft can update the roots as they please (and historically issued updates up unto Windows 2000, so root updates are a demonstrable solved problem).
BSDs and Linuxes: (Usually) Uses Mozilla's trust list. Also updates separately from system updates, so unless you stick somehow with unsupported systems you already have this (and can be manually included into the trusted roots if you insist on using that outdated version).
macOS: bundled with the system updates (see caveat with Firefox independently managing roots). Here, I don't know what version is the oldest one with ISRG root certs.
Natively? Windows 7, assuming you have installed all updates before January. Microsoft can update the roots as they please (and historically issued updates up unto Windows 2000, so root updates are a demonstrable solved problem).
BSDs and Linuxes: (Usually) Uses Mozilla's trust list. Also updates separately from system updates, so unless you stick somehow with unsupported systems you already have this (and can be manually included into the trusted roots if you insist on using that outdated version).
macOS: bundled with the system updates (see caveat with Firefox independently managing roots). Here, I don't know what version is the oldest one with ISRG root certs.
[deleted]
The real story here is the state of Android and the devices it runs on.
Nice TL;DR, thx
I don't understand the motivation for making this change now. Why not keep the universally accepted root certificate as the default chain? Why does Let's Encrypt need to switch to their own root certificate now, and cause thousands of websites to break on older devices?
I don't even control the certificate provisioning process. We use Heroku and Webflow. This is frustrating.
I don't even control the certificate provisioning process. We use Heroku and Webflow. This is frustrating.
Their cross cert expires in 10 months. Switching in January gives most people time to notice the problem while there's an easy temporary fix (switch to the soon to be expiring cross cert while you evaluate the root compatability of commercial CAs across the devices you support).
I imagine the cross cert cost a bunch of money, and they may not have the money to do that again.
I imagine the cross cert cost a bunch of money, and they may not have the money to do that again.
Did you miss the part where the "universally accepted" root certificate is going to be universally rejected in 10 months?
"the DST Root X3 root certificate that we relied on to get us off the ground is going to expire - on September 1, 2021."
What I have never understood is why IdenTrust accepted to cross-sign Let’s Encrypt's root certificate.
With that move, IdenTrust basically broke the CA cartel and helped driving the price of basic certificates to zero. How did they, as a for-profit organization, justify "doing the right thing" when that meant disrupting their own market?
Anyway, kudos to IdenTrust.