Mosquitto CRL Revocation: Why crlfile Rejects Every Client

You have a working Mosquitto mTLS listener. Devices present client certificates from your private CA, require_certificate true is on, everything connects. Then you add one line so that revoked devices actually stop working:

crlfile /mosquitto/certs/crl.pem

And now nothing connects. Every device — including ones you have never revoked — fails the handshake with some variation of:

OpenSSL Error[0]: error:0A000086:SSL routines::certificate verify failed
client sent: tlsv1 alert unknown ca

Search for this and you will find mosquitto#2597 ("All connections rejected if crlfile is used"), mosquitto#3352 (unknown ca with an intermediate in the cafile), and mosquitto#2916 (OpenSSL errors after a reload). The advice that consistently works is: remove the crlfile line.

That advice is not a fix. It is shipping a fleet with no revocation, which means a leaked device key stays valid until the certificate expires and your only recourse is deleting the device. Every PKI vendor blog will tell you to "have a revocation strategy"; almost none of them show a broker config that survives contact with OpenSSL.

This post is that config. There are four separate traps here, and they compound — fixing one just moves you to the next error, which is exactly why the problem gets abandoned. If you are terminating device mTLS in Node.js as well, there is a fifth one waiting for you.

Not set up for device certificates yet? This post is the direct sequel to Mosquitto mTLS Tutorial: Per-Device Client Certificates with a Private CA, which builds the broker, the CA and the per-device certificates assumed below, from scratch. Start there if require_certificate and use_identity_as_username are not already working for you — the tutorial ends at precisely the point this post begins.

The setup throughout is step-ca as the private CA (root → intermediate → short-lived device leaves) and Mosquitto as the broker, but the traps are OpenSSL's, not step-ca's. They apply to any CA with an intermediate.

First, what OpenSSL is actually doing

When you set crlfile, you are not just handing Mosquitto a list of bad serials. You are switching on an entire additional verification pass, and that pass has its own requirements that the plain (no-CRL) path never had.

For each certificate it is asked to check, OpenSSL must:

1. Build a chain from the leaf to a trusted root.
2. Find a CRL whose scope covers that certificate.
3. Verify the CRL's signature — which means having the CRL issuer's certificate available.
4. Confirm the CRL is currently valid — inside its thisUpdate/nextUpdate window.
5. Only then, check whether the serial is on it.

Steps 2, 3 and 4 are new. Each is a place the handshake can now fail for a certificate that verified perfectly fine yesterday. And critically, all of these failures surface as the same generic verify error, which is why the symptom (unknown ca for every client) tells you almost nothing about which one you hit.

Four gates that only exist once CRL checking is enabled. All four report as a generic verification failure.

Trap 1: your trust store needs the intermediate, not just the root

This is the one that bites first and confuses most.

Without CRL checking, a root-only cafile works fine. The client sends its leaf and the intermediate in the handshake, OpenSSL builds the chain from what it was given, anchors it at your trusted root, and everyone is happy.

Turn on CRL checking and that breaks — because the CRL itself is signed by the intermediate, not the root. step-ca holds the intermediate key and uses it to sign the CRL of revoked leaves. To verify that signature in step 3 above, OpenSSL needs the intermediate's certificate in the trust store, not merely passed along in the handshake.

So the fix is to make the trust store the full chain:

cat intermediate_ca.crt root_ca.crt > /mosquitto/certs/root_ca.crt

cafile /mosquitto/certs/root_ca.crt   # intermediate + root, concatenated

Order matters to some tooling and not to others; leaf-most first (intermediate, then root) is the conventional and safest ordering.

If you take one thing from this post: the file your cafile points at is not "your root CA", it is "every CA certificate OpenSSL might need" — and once CRLs are involved, that set grows to include whichever CA signs your CRLs.

Trap 2: "different CRL scope"

Fix trap 1 and you may land on this, which is far more obscure:

OpenSSL Error: different CRL scope

This one is about a pair of X.509 extensions that have to agree:

• A CRL can carry a critical Issuing Distribution Point (IDP) extension, declaring the scope of certificates it covers.
• A certificate can carry a CRL Distribution Points (CDP) extension, declaring where its revocation info lives.

When a CRL has an IDP, OpenSSL will only apply it to certificates whose CDP matches that scope. step-ca stamps a critical IDP on the CRL it serves. If your device certificates carry no CDP at all, nothing matches — so OpenSSL concludes it has no CRL covering this certificate, and rejects it. Every certificate fails, for the same reason, and the message points at scope rather than at anything you would recognise as your own misconfiguration.

The fix is to make issued leaves carry a CDP that matches. In step-ca that means giving the device provisioner an x509 template:

{
	"subject": {{ toJson .Subject }},
	"sans": {{ toJson .SANs }},
	"keyUsage": ["digitalSignature"],
	"extKeyUsage": ["serverAuth", "clientAuth"],
	"crlDistributionPoints": ["https://your-ca.example.com/1.0/crl"]
}

and pointing the provisioner at it in ca.json:

{
  "name": "helix-device",
  "type": "JWK",
  "options": { "x509": { "templateFile": "/path/to/device-leaf.tpl" } }
}

The URL must match the IDP step-ca puts on the CRL. Note the consequence: certificates issued before you added the template have no CDP and will never validate against an IDP-scoped CRL. This is a reissue-the-fleet change, not a hot fix — worth knowing before you roll it out to production devices.

Trap 3: CRL_CHECK_ALL wants a CRL for the intermediate too

OpenSSL has two CRL modes, and which one you get depends on the software driving it:

Mosquitto uses the leaf-only mode, so a broker-only deployment needs just step-ca's CRL. But Node.js's crl TLS option turns on CRL_CHECK_ALL — and under that flag, OpenSSL also demands a CRL covering the intermediate, which would be issued by the root.

step-ca does not serve one. It publishes the intermediate-signed CRL of revoked leaves and nothing else. So the moment a Node mTLS listener joins the picture, verification fails on a missing CRL you did not know you needed.

The workaround is to generate an empty, long-lived root-signed CRL once, at PKI setup, and concatenate it onto the one you distribute:

WORK=$(mktemp -d)
: > "$WORK/index.txt"
echo 1000 > "$WORK/crlnumber"
cat > "$WORK/openssl.cnf" <<EOF
[ca]
default_ca = CA_default
[CA_default]
database = $WORK/index.txt
crlnumber = $WORK/crlnumber
default_md = sha256
default_crl_days = 3650
EOF

openssl ca -gencrl -config "$WORK/openssl.cnf" \
  -cert  certs/root_ca.crt \
  -keyfile secrets/root_ca_key \
  -passin "file:secrets/password.txt" \
  -out   certs/root_crl.pem

It is empty because you are not revoking your own intermediate; it exists purely to satisfy CRL_CHECK_ALL. Ten-year validity keeps it out of trap 4. Mosquitto ignores it harmlessly, so you can ship one concatenated CRL file to both enforcers.

One further Node-specific detail: given a buffer containing several PEM CRLs, Node loads only the first. You have to split the file and pass an array:

const crls = pem.split(/(?=-----BEGIN X509 CRL-----)/g).filter((s) => s.trim());
// pass `crls` (an array), not the raw concatenated buffer

Trap 4: an expired CRL fails closed, and that is correct

CRLs expire. Each carries a nextUpdate, and TLS implementations are expected to treat an expired CRL as a hard failure rather than quietly skipping revocation — Mosquitto does exactly that. This is the right behaviour, and it is also a self-inflicted outage waiting to happen: a CRL that lapses for thirty seconds rejects your entire fleet for thirty seconds.

Two values govern this and they are easy to set inconsistently:

• step-ca's cacheDuration — both the CRL's validity window and how often step-ca proactively regenerates it.
• your distribution interval — how often you actually fetch and stage the new CRL.

{ "crl": { "enabled": true, "cacheDuration": "2m" } }

The rule is simple: cacheDuration must be comfortably larger than your sync interval. We run cacheDuration at 2 minutes against a 15-second sync — roughly 8× headroom, so a couple of failed fetches in a row still cannot leave a staged CRL past its expiry. Setting them equal, or syncing slower than the CRL lives, guarantees intermittent fleet-wide rejections that look random.

The same value also sets your revocation latency: a revoked serial reaches enforcement within about one cacheDuration. Shortening it tightens that window but shrinks your expiry safety margin — that is the actual trade-off to reason about.

Trap 5: don't put a CRL in a Node.js SecureContext

If you also terminate device mTLS in Node, there is one more, and this one has no upstream issue we could find.

Passing crl into a TLS SecureContext on Node 24 / OpenSSL 3.5 corrupts the connection as soon as you read getPeerCertificate(). The handshake completes normally, the request is served, and then the next request on that connection dies with unexpected EOF. Because it needs both the CRL and a peer-certificate read to trigger, it looks like a flaky keep-alive problem rather than a TLS configuration one.

We stopped fighting it and moved revocation up a layer. The TLS context verifies the chain; the application checks the serial:

// Chain verification only — no `crl` here.
createHttpsServer({ ca, cert, key, requestCert: true, rejectUnauthorized: true }, handler);

// Revocation, enforced per-connection against our own records.
const serial = socket.getPeerCertificate()?.serialNumber?.toUpperCase();
if (serial === null || (await isDeviceCertRevoked(db, serial))) {
  socket.destroy();
  return;
}

This is arguably better anyway. You already need a database record of every issued certificate to build an admin UI, an audit trail, or an expiry report. Once that table exists, revoked_at IS NOT NULL is a cheaper and far more debuggable check than parsing a CRL on every handshake — and it takes effect the instant you revoke, with none of the cacheDuration latency.

The CRL remains the enforcement mechanism at the broker, where it works well. Use each where it behaves.

Diagnosing this without breaking anything

Every command here is read-only. Run them against a live broker safely.

Does the CRL actually parse, and when does it expire?

openssl crl -in /mosquitto/certs/crl.pem -noout -text | head -20

Check Next Update against the current time, and look for the X509v3 Issuing Distribution Point block — its presence is what activates trap 2.

Does your device certificate carry a matching CDP?

openssl x509 -in device.crt -noout -text | grep -A3 "CRL Distribution Points"

Empty output on an IDP-scoped CRL means trap 2.

Verify the whole thing offline, exactly as OpenSSL will:

# Leaf-only, what Mosquitto does
openssl verify -crl_check \
  -CAfile /mosquitto/certs/root_ca.crt \
  -CRLfile /mosquitto/certs/crl.pem \
  device.crt

# Whole chain, what Node does
openssl verify -crl_check_all \
  -CAfile /mosquitto/certs/root_ca.crt \
  -CRLfile /mosquitto/certs/crl.pem \
  device.crt

This is the highest-value command in the post. It reproduces the broker's decision with a specific error message instead of a generic TLS alert, and the difference between the two invocations tells you immediately whether you are looking at trap 3.

Watch a real handshake:

openssl s_client -connect broker.example.com:8883 \
  -cert device.crt -key device.key \
  -CAfile root_ca.crt -showcerts

Confirm what the CA is serving (step-ca returns DER, so convert before reading):

curl -fsS --cacert root_ca.crt https://127.0.0.1:9000/crl -o crl.der
openssl crl -inform DER -in crl.der -noout -text | head -20

The working configuration

Putting it together. The device listener enforces revocation; a separate service listener for internal components with long-lived, directly-issued certificates does not need to:

# Device MQTT over TLS — CRL enforced.
listener 8883
protocol mqtt
allow_anonymous false
require_certificate true
use_identity_as_username true
cafile /mosquitto/certs/root_ca.crt   # intermediate + root (Trap 1)
certfile /mosquitto/certs/server.crt
keyfile /mosquitto/certs/server.key
crlfile /mosquitto/certs/crl.pem      # step-ca CRL + root CRL (Trap 3)
tls_version tlsv1.2

use_identity_as_username true is worth calling out: it promotes the client certificate's CN to the MQTT username, so your ACL file can authorise per device off the cryptographic identity rather than a parallel password database.

The distribution side is a small loop — fetch, normalise DER to PEM, append the root CRL, install, and signal the broker on change:

curl -fsS --cacert "$ROOT_CA" "$CA_URL/crl" -o "$der"
openssl crl -inform DER -in "$der" -outform PEM -out "$pem"
cat "$ROOT_CRL" >> "$pem"                       # Trap 3
install -m 0644 "$pem" /mosquitto/certs/crl.pem

# Only reload when the content actually changed.
systemctl reload mosquitto.service

Mosquitto re-reads the crlfile on SIGHUP, so revocations apply without dropping existing connections.

Two operational rules matter as much as the config:

Fail closed on startup. Stage a CRL before the listener binds, and refuse to start if you cannot get one. A broker that comes up without a CRL is a broker silently not checking revocation — the exact failure you are trying to design out.

if [[ ! -f /mosquitto/certs/crl.pem ]]; then
  echo "no CRL available; refusing to start" >&2
  exit 1
fi

Keep certificates short-lived regardless. We issue device certificates with a 24-hour default and a 168-hour ceiling. Short lifetimes are the backstop that makes the whole scheme forgiving: if CRL distribution breaks, your exposure is bounded by the TTL rather than unbounded. The CRL closes the gap within that window; it is not a substitute for having a small window.

Wrap-up

The reason "just remove crlfile" became the accepted answer is that four distinct failures all present as one generic verification error, and each fix only reveals the next. Worked through in order they are quite tractable:

1. Put the intermediate in your trust store — it signs the CRL, so OpenSSL needs it to verify one.
2. Stamp a CDP on issued leaves matching the CRL's IDP, or scope matching rejects everything.
3. Supply an empty root-signed CRL for the intermediate if anything in your stack uses CRL_CHECK_ALL.
4. Keep cacheDuration well above your sync interval so a CRL never lapses in production.

And if you terminate mTLS in Node, skip the TLS-layer CRL entirely and check revocation against your own certificate records — you almost certainly want that table anyway.

The payoff is a fleet where revoking a device actually takes effect in minutes, at the broker, without deleting anything or waiting out a certificate lifetime. That is worth considerably more than the afternoon these four traps will cost you.

Starting from scratch? Mosquitto mTLS Tutorial: Per-Device Client Certificates with a Private CA builds the broker, private CA and per-device certificates this post assumes.