Commit Graph
9 Commits
Author SHA1 Message Date
jcoffey-dev 63fe315457 Specs: ufw is in the way of the ACME test, and the rules say so
/etc/ufw/user.rules is world-readable, so this needed no privilege after
all. The default input policy is DROP and 8899 is not among the allowed
ports, so pebble's connection to the suite's listener is dropped. That
matches the live run, where twelve probes from a container completed no
handshake while the same openssl reached pebble's own TLS port.

The nc readings that pointed the other way — instant refusals on closed
ports, where a DROP should hang — are still unexplained, and are left on
the page as unexplained rather than quietly dropped, since they are what
sent an earlier pass through this page in the wrong direction.

Nobody has added the allow rule and re-run the suite, so the fix is
written down as a prediction. The cutover draft says the same: the test
failing here is not evidence that renewal works on the host.
2026-09-19 21:27:40 -07:00
jcoffey-dev 2a127d6b9a Specs: the ACME network question is open, not answered
Probing further contradicted the previous two commits. During a live run,
when the suite is certainly listening on 8899, twelve handshakes from a
container completed nothing, with or without the ACME ALPN, while the same
openssl in the same container talks to pebble's TLS port and prints its
certificate. A listener that is up but unreachable from a container is what
a ufw DROP looks like. An instant refusal on a closed port, which nc saw
from two images, is not. Both were observed minutes apart.

So the ufw suspicion is neither confirmed nor dismissed, and the page now
says that rather than picking the reading that suits the last probe.
Settling it needs `sudo ufw status verbose` and a listener bound by hand,
neither of which this session could do.

What stands on pebble's own log, and does not depend on any of this: it
runs validations and marks the authorizations invalid, so "never validates,
stays pending" was wrong.
2026-09-19 18:10:22 -07:00
jcoffey-dev 261175aae5 Specs: say only what the ACME probes support
The previous commit called the bridge open on the strength of one nc run.
A later openssl s_client against the same closed port hung for its whole
timeout instead of reporting the refusal nc had just seen. Repeating the nc
test from a second image, with a control port and two closed ports, agreed
with the first: immediate refusal, which is not what a DROP looks like. The
openssl behavior is still unexplained, so the claim now carries what was
measured and the anomaly beside it.

The ALPN probe is recorded as proving nothing, for the same reason: it
hangs against a port with nothing behind it, so its silence during a
renewal says nothing about acme-tls/1.

Pebble's log is untouched by any of this: it validates and marks the
authorizations invalid, which is what makes the old explanation wrong.
2026-09-19 18:06:10 -07:00
jcoffey-dev 4336251cea Specs: the ACME failure isn't ufw, and pebble does validate
The page blamed ufw for blocking the docker bridge, and said pebble never
validates, so the authorizations stay pending. All three are wrong.

From a container on the ACME network the host answers on both gateway
addresses: port 22 connects, and 8899 refuses at once with nothing
listening, where a DROP would hang. No firewall rule was read or changed to
establish that. Pebble's own log shows 20 validation attempts in the
regression run, five for each of the four tls.org names, each ending in
"INVALID by completed challenge". The challenges are answered and refused.

So the order goes invalid, no certificate is issued, and the test unwraps a
None. What's left to explain is the TLS-ALPN handshake. The responder is
intact and listen.rs picks it per connection from has_acme_tls_challenge,
which is computed when the network config is parsed, while the test adds
its provider after boot. That's written down as a hypothesis, not a
finding: it hasn't been tested.
2026-09-19 17:57:52 -07:00
jcoffey-dev 08b3210cff Specs: the regression after the ACME fix, 87 passed and 3 failed
Ran it single-threaded on RocksDb: 87 passed, 3 failed, 23 ignored, all 113
tests a default build holds. lmtp_delivery passed this time, which is what
the note about queue timing under a sequential run predicted, so the four
documented failures are three. The ACME suite logged no 400 at all, so the
renewal fix holds; it still ends on the certificate that never came,
because pebble can't validate across the bridge with ufw up.

The line it replaces claimed 86 passed, 4 failed, 27 ignored from the same
command. That totals 117, and no feature set of this tree produces 117 —
113 by default, 118 with all three backends — with no test added or removed
since. Said so rather than presenting the two as a trend.
2026-09-19 17:21:58 -07:00
jcoffey-dev adfa22c817 Specs: the nine ignored suites re-run, and the STORE each one needs
Re-ran all nine on containers removed beforehand, one suite at a time.
All nine pass. mysql_replica_position_tests failed the first time on test
18's lag assertion, with the pair half a minute old and still catching up,
and passed on a second run against the same containers; it fails before
the point where it changes the replica's settings, so it leaves nothing to
restore. Noted both, with each suite's time.

The table's STORE column said "default" for five suites, which reads as
"leave it unset". The harness has no default: it panics with "Missing or
invalid store type" before the suite starts. They run on RocksDb, as the
regression does, so the column now names it.
2026-09-19 16:54:27 -07:00
jcoffey-dev 309835be1d ACME: post a challenge once, then poll the authorization
The renewal loop re-posted the challenge every time it polled and found the
authorization still pending. RFC 8555 section 7.5.1 has the client post a
challenge once to say it is ready and then poll; a server that has already
moved the challenge to "processing" refuses a second post, and pebble
answers 400 malformed, which failed the whole renewal.

Verified against pebble: the 400s are gone and the client polls. The suite
still can't finish on this machine, because pebble never reaches the test
server to validate the challenge; that path is the environment, and the
runbook now says so.
2026-09-19 16:28:15 -07:00
jcoffey-dev ce12659138 Specs: what a plain regression leaves failing, and why none of it is ours 2026-09-19 16:15:09 -07:00
jcoffey-dev e492b7875a Specs: how to run the suites a plain regression doesn't reach
`cargo test -p tests` runs none of the fork's own feature suites except
what's inside `system_tests`: SCIM, per-domain directories, the sharded
stores and the four replica suites are all ignored, each needing a
container, a STORE the harness only builds on request, or both. The
runbook lists what each one needs, and the container-reuse trap.
2026-09-19 16:14:44 -07:00