Who do you trust when you trust a certificate? The eMudhra incident explained

In October 2026, a researcher found out that eight intermediate certificates issued by eMudhra (a certificate authority, one of the ~140 organizations whose signature your browser and operating system accept as proof that a website is who it claims to be) had private keys that anyone could recover from the public certificate. One of them was the number 4. Yes, one digit. Four. Not a 384-bit secret, not a weak random generator, just the number you count to after three. Browsers blocked the certificates in a few hours. If you run anything that is not a browser, like a Python service, a Go binary, a Node worker, or the curl in your CI (continuous integration) pipeline, you probably kept trusting them for days, and you may still be trusting them right now. And while you dispatch your coding agent to take care of it, it’s worth understanding what problem it is actually solving, because this one hides in more places than a lock file.

This is an overview of what a TLS (Transport Layer Security) connection trusts, what went wrong at eMudhra, why revoking the certificates didn’t help most of us, and what you should check in your own containers. You don’t need any cryptography background to follow it; if you know what the padlock in your browser means, you have what it takes. The examples are Python, because that’s where I hit the problem, but the problem itself is about which files your runtime trusts, so if you write Go, Node, Java, or Ruby you should be able to map every step to your own stack. Some parts of this story are already well explained elsewhere, so I’ll keep those short and leave reading tips along the way for whoever wants to go deeper than a blog post can.

What does TLS trust, anyway?

TLS gives a connection two things. It encrypts, so nobody on the path can read what you send, and it authenticates, so you know the party you’re encrypting to is the server you meant to reach. We usually think about the first one, but the second one is where this story lives. Encrypting to the wrong party is worse than not encrypting at all, because it feels safe.

Authentication rides on the certificate the server sends during the handshake. Your client checks that certificate before agreeing on the session keys. If the check passes, the padlock closes. If it passes when it shouldn’t, the padlock closes anyway.

Step 3 is the whole story. Everything after it assumes the certificate was genuine.

The handshake itself is well documented, so I’m not going to replicate it here.

Further reading

What happens in a TLS handshake? is Cloudflare’s readable walkthrough of the five steps above.

Transport Layer Security on MDN (Mozilla Developer Network) is the reference, with the version history.

RFC 8446 is TLS 1.3 itself, if you prefer the source of truth. RFC stands for Request for Comments, the series where internet standards are published.

How does a root certificate become something your connection trusts?

A certificate is a small signed document: a name, a public key, a validity window, and a signature from whoever issued it. The server’s own certificate, the leaf, is signed by an intermediate certificate authority (CA). The intermediate is signed by a root. The root signs itself, which sounds like cheating, and it is, on purpose.

Your client walks that chain upwards. At each step, it takes the issuer’s public key and checks the signature on the certificate below. The walk ends when it reaches a certificate that is already in its trust store, a file with roughly 140 roots that came with your browser, your operating system or your language runtime. Being in that file is what "trusted" means. There is no further authority to ask.

The leaf is trusted because the intermediate vouches for it, and the intermediate is trusted because its root sits in the store.

Notice one thing in the picture: nothing in the chain says which hostnames the intermediate is allowed to vouch for. A CA certificate is not scoped to a domain. Any intermediate under any trusted root can sign a leaf for any hostname on the internet, and the chain will verify. If you want to look at a real chain, Let’s Encrypt publishes its own, cross-signs and all.

Further reading

How Let’s Encrypt works explains how a CA decides to sign your leaf in the first place, domain validation included. It is the best short tutorial on the issuing side I know of.

RFC 5280, section 6 is the validation algorithm your client runs, step by step.

Mozilla’s Root Store Policy is what a CA has to comply with to be one of those ~140 roots. Surprisingly readable.

Where does trust live?

"The trust store" sounds like one thing. On a real machine, it is several files that happen to agree most of the time. Your browser has its own. The operating system has one; on Debian it is the ca-certificates package. Python’s ssl module reads the operating system file by default, but httpx and requests read a Python package called certifi instead. pip carries a private copy of certifi. uv, written in Rust, bundles its own list. grpc compiles a roots.pem into its wheel. Java has cacerts, Node has its own bundled roots, and so on.

One Python container I looked at. Five copies of the same root, each with its own release train.

All of them descend from Mozilla’s root program, so on a quiet day they contain the same roots. On a loud day, each one is a separate fix with a separate owner. Fixing certifi fixes the first row of that picture and nothing else. Keep this image in mind — we’ll come back to it.

The key was 4

The intermediates in this story used elliptic-curve (EC) keys on the P-384 curve. An EC private key is a number, let’s call it d, chosen at random from about 2^384 possible values. The public key is Q = d · G, where G is a fixed point on the curve. Going from d to Q is cheap. Going back from Q to d is infeasible, as long as d is random. That’s all the curve maths this post needs.

That guarantee has a precondition the maths cannot check for you: d has to actually be random. On April 30th, 2025, eMudhra ran a key generation ceremony for its new emSign intermediates. By the company’s own account on the Mozilla bug tracker, a flaw in the ceremony code produced a second set of keys that were weaker, and because the key labels looked alike, the weak set ended up being used to issue the intermediates and to request cross-signs from Sectigo.

Weaker is a gentle word. The top 376 bits of each private key were zero. The entire secret lived in the last byte, so d was below 256 for all eight keys. For one of the TLS intermediates it was exactly 4. Since the public key is printed in the certificate, anyone can multiply G by 1, 2, 3, 4 and compare.

Nothing was stolen and no hardware security module was breached. The secret was just small enough to guess.

The recovery fits in a few lines. This is the shape of the check Mozilla ran to recover the keys themselves, within two hours of the report, written against the public key found in the certificate:

from cryptography.hazmat.primitives.asymmetric import ec

target = intermediate_cert.public_key().public_numbers()
for d in range(1, 256):
    candidate = ec.derive_private_key(d, ec.SECP384R1()).public_key()
    if candidate.public_numbers() == target:
        print("private key is", d)  # prints 4 for EM OV TLS CA - G3C
        break

With d in hand, an attacker can sign anything the intermediate could sign, which means a leaf certificate for any hostname on earth, chaining cleanly to a root in your store.

Further reading

A relatively easy to understand primer on elliptic curve cryptography is the one I’d recommend for the maths I skipped.

A CA private key was the number 4 walks through the recovery with a working script and even a mitmproxy demo, if you want to see it end to end.

emudhra.we-have-a-problem.com is the researcher’s disclosure site, with signed proofs for all eight keys and test endpoints you can curl.

What’s eMudhra?

eMudhra Limited is a publicly listed certificate authority that has been licensed by its national regulator since 2008 to issue digital signature certificates. For the public web it runs the emSign brand, with roots accepted by the Mozilla, Microsoft and Apple root programs. Its roots entered certifi with the 2019.3.9 release, which is why the advisory’s affected range starts there.

If you like post-mortems, here is the short version of this one, all from the Mozilla bug:

• Impact – Eight intermediate CAs with recoverable keys: six for TLS under emSign ECC Root CA – G3, two for code signing, several of them cross-signed by Sectigo under the old AAA Certificate Services root. Around 30,000 live certificates chain to the root that is now being removed.
• Root cause – A flaw in the April 2025 key ceremony code generated a second, weak key pair on the HSM (hardware security module, the tamper-resistant box a CA keeps its keys in). Similar key labels led staff to use the weak pair. The HSM did its job; the code around it did not, and no later review caught it.
• Detection – Seventeen months later, by an outside researcher, from the public certificates alone. An earlier bug about invalid CRL (certificate revocation list) signatures on the same intermediates had already touched those keys, and nobody followed the thread.
• Response – All eight revoked within 14 hours of the report. Browsers blocked them the same night. Mozilla then asked eMudhra to stop issuing under the G3 root and moved to remove it from the Mozilla store on an emergency basis; Chrome plans the same.

Eight days before the disclosure, Mozilla had closed the public discussion on approving four new eMudhra roots. Timing is everything.

Revoked on Saturday, trusted by your script on Tuesday

Revocation is the public key infrastructure’s (PKI) answer to a compromised key. The CA publishes "this certificate is no longer valid", through a CRL or OCSP (Online Certificate Status Protocol), and clients are supposed to check. Browsers do. Firefox consumes OneCRL and CRLite, Chrome consumes CRLSets, and both get pushed to users within hours. Browsers also enforce Certificate Transparency: a leaf has to carry proof that it was publicly logged, and these intermediates had never logged a single certificate. Either check alone would have rejected a forged leaf.

Almost nothing outside a browser does either. Python’s ssl, Go’s crypto/x509, Node, Java, curl and OpenSSL’s defaults build the chain, look for the root in their file, and stop. No CRL (downloading the CA’s list of revoked certificates), no OCSP (asking the CA whether this specific certificate is still good), no transparency check (looking for proof that the certificate was logged publicly). This is not laziness, by the way: online revocation checks are slow, leak which sites you visit, and fail open when the responder is down, which is why Let’s Encrypt stopped serving OCSP altogether in 2025. For them, eMudhra’s revocation on October 18th changed nothing. A forged leaf signed with d = 4 validated on Monday exactly as it did on Friday, and would keep validating until the file itself changed.

Same forged certificate, same root. The browser has two independent reasons to refuse it; the runtime has none.

Was it exploited? Nothing forged from these keys has shown up in Certificate Transparency logs or been reported in use. That’s weaker reassurance than it sounds, since an attacker going after non-browser clients has no reason to log anything. The attacker also still needs to be on the path, with a hijacked DNS (Domain Name System) answer, a hostile Wi-Fi or a compromised proxy. Then your padlock closes on them.

Further reading

CRLite: all of the Web PKI’s revocations, compressed explains how Firefox ships every revocation to every user in a few hundred kilobytes.

CRLSets is the Chromium equivalent, and the page is honest about what it does not cover.

certificate.transparency.dev is the starting point for how Certificate Transparency logs work and why browsers demand them.

Two ways to fix it

If your client’s trust is a flat PEM file (Privacy-Enhanced Mail, the plain-text format certificate bundles still use), there is no way to say "stop trusting these eight intermediates". The file lists roots and nothing else. The only lever you have is removing a root, and everything beneath it goes with it. That’s what certifi, Debian and Red Hat with their ca-certificates packages, and uv did: emSign ECC Root CA – G3 and the Comodo AAA root are gone from the bundle, so a forged chain has no anchor to land on. certifi‘s first attempt, 2026.10.18, removed every emSign root; 2026.10.19 put the unaffected ones back, which is why that’s the version to pin. Python itself shipped 3.13.17, 3.14.9 and 3.15.1 just to refresh the certifi copy inside the bundled pip.

A verifier that is code rather than a file can be surgical. Go added a denylist of the eight public keys inside crypto/x509; the root stays trusted, but any chain passing through one of those intermediates fails. Node did the same and refreshed its roots. The difference is collateral damage. Removing the root invalidates roughly 30,000 legitimate certificates that now have to be reissued under another CA. A denylist invalidates exactly the eight keys that were broken.

A PEM file can only lose a root. A verifier with code can lose one key. Both end in "rejected"; they differ in who else gets hurt.

Further reading

GHSA-6q48-6c7w-9gw2 is the certifi advisory: affected range, the two releases, and why you want the second one.

DSA-6572-1 is the Debian Security Advisory (DSA) for ca-certificates, with the exact package versions per release.

GO-2026-6809 describes the Go denylist and the fixed versions.

As a developer, what does this mean for me?

Go back to the container picture. Everything I had to do in the services I work on comes from it, and I suspect your list will look alike.

Count your trust stores. Grep your images for cacert.pem, roots.pem and /etc/ssl/certs, and remember the binaries that don’t use any of them: uv, Go programs and Node ship their own lists inside the executable. One Python container I checked had five stores. A lock file bump fixes one of them.

Bump certifi to 2026.10.19 and rebuild. If Dependabot opened the PR for you, good, but don’t count on it: the certifi advisory has no CVE (Common Vulnerabilities and Exposures) identifier, and in one repository I follow the alert only showed up days after the fix was out. A lock file bump also does nothing for a container that is already running. You need the image rebuilt and the deployment rolled.

Check how your base image gets its operating system store. If you pin python:3.13.13-slim-bookworm, you froze Debian too: official images only rebuild the tags that are still current, so Debian’s fix will never reach that one. You have to move to 3.13.17. Floating tags are not safe either, just in a different way. The ghcr.io/astral-sh/uv:python3.13-bookworm-slim images stopped being built in February 2026 when Astral moved to Debian Trixie, and in October they were still shipping uv 0.9 and a trust store with the compromised root. A tag that nobody builds anymore looks exactly like a tag that is up to date.

Don’t forget the build-time stores. uv downloads your wheels with its own root list; 0.13.4 is the release that dropped the root. If your lock file carries hashes, a forged index could at most break your build, not swap a package, but you still want the upgrade.

And the ones with no release yet. grpc bundles a roots.pem in its wheel and, at the time of writing, had updated it on master without publishing a release. If your service talks gRPC to the outside world, GRPC_DEFAULT_SSL_ROOTS_FILE_PATH lets you point it at the operating system store in the meantime.

Finally, test it. The researcher’s disclosure site hosts endpoints whose certificates chain through the compromised intermediates. A curl from inside your container that fails with a certificate error is what you want to see.

Further reading

uv’s Docker guide lists which image tags are still built, and how to copy just the uv binary into a base image you control.

docker-library/official-images documents how official image tags are defined and when they get rebuilt.

gRPC environment variables covers GRPC_DEFAULT_SSL_ROOTS_FILE_PATH and the other knobs that change where gRPC looks for roots.

Conclusion

The curve maths was fine. The key ceremony was not, and no later check could tell, because randomness is a precondition of the whole system, not a property you can verify afterwards. Revocation turned out to protect browsers and almost nobody else, because trust for the rest of us is a file, and files only change when someone rebuilds the image. The fix is boring, which is a good thing: count your stores, bump what has a release, rebuild what pins a tag, redeploy, and go check what else in your containers quietly stopped receiving updates.

We want to work with you. Check out our Services page!