Specs: the regression after the ACME fix, 87 passed and 3 failed
Ran it single-threaded on RocksDb: 87 passed, 3 failed, 23 ignored, all 113 tests a default build holds. lmtp_delivery passed this time, which is what the note about queue timing under a sequential run predicted, so the four documented failures are three. The ACME suite logged no 400 at all, so the renewal fix holds; it still ends on the certificate that never came, because pebble can't validate across the bridge with ufw up. The line it replaces claimed 86 passed, 4 failed, 27 ignored from the same command. That totals 117, and no feature set of this tree produces 117 — 113 by default, 118 with all three backends — with no test added or removed since. Said so rather than presenting the two as a trend.
This commit is contained in:
@@ -90,9 +90,19 @@ STORE=RocksDb cargo test -p tests -- --exact automation::automation_tests
|
|||||||
|
|
||||||
## What a plain regression leaves failing
|
## What a plain regression leaves failing
|
||||||
|
|
||||||
`STORE=RocksDb cargo test -p tests -- --test-threads=1` was 86 passed, 4
|
`STORE=RocksDb cargo test -p tests -- --test-threads=1` was 87 passed, 3
|
||||||
failed, 27 ignored on 2026-09-19. None of the four was a fork regression,
|
failed, 23 ignored in 17m 26s on 2026-09-19, the run after the ACME fix.
|
||||||
and three are the invocation or the environment rather than the code:
|
That is every one of the 113 tests a default build holds; the five the
|
||||||
|
`postgres`, `mysql` and `redis` features add are all in the table above and
|
||||||
|
all `#[ignore]`d. None of the three failures is a fork regression: each is
|
||||||
|
the invocation or the environment rather than the code.
|
||||||
|
|
||||||
|
An earlier run the same day was recorded here as 86 passed, 4 failed, 27
|
||||||
|
ignored. That totals 117, which no feature set of this tree produces — 113
|
||||||
|
by default, 118 with all three backends — and no commit since has added or
|
||||||
|
removed a test. So the two aren't the same build and shouldn't be read as a
|
||||||
|
trend; the numbers above are the reference, with the command that produced
|
||||||
|
them.
|
||||||
|
|
||||||
- `smtp::inbound::antispam::antispam` fails every time. Its rules URL
|
- `smtp::inbound::antispam::antispam` fails every time. Its rules URL
|
||||||
defaults to `file:///Users/me/code/spam-filter/spam-filter-rules.json.gz`,
|
defaults to `file:///Users/me/code/spam-filter/spam-filter-rules.json.gz`,
|
||||||
@@ -102,18 +112,21 @@ and three are the invocation or the environment rather than the code:
|
|||||||
- `cluster::broadcast::cluster_tests` needs `COORDINATOR=Redis` (or `Nats`)
|
- `cluster::broadcast::cluster_tests` needs `COORDINATOR=Redis` (or `Nats`)
|
||||||
and a store the nodes can share: it passes on `STORE=PostgreSql`, and
|
and a store the nodes can share: it passes on `STORE=PostgreSql`, and
|
||||||
can't work on RocksDb, where each node gets its own.
|
can't work on RocksDb, where each node gets its own.
|
||||||
- `smtp::outbound::lmtp::lmtp_delivery` counted three DSNs where it wanted
|
- `smtp::outbound::lmtp::lmtp_delivery` once counted three DSNs where it
|
||||||
four, and passes on its own: queue timing under a loaded sequential run.
|
wanted four, and passed in this run: queue timing under a loaded
|
||||||
- `automation::automation_tests` failed against pebble with
|
sequential run, not a fault to chase.
|
||||||
|
- `automation::automation_tests` had failed against pebble with
|
||||||
`400 malformed: "Cannot update challenge with status processing, only
|
`400 malformed: "Cannot update challenge with status processing, only
|
||||||
status pending"`, because the client re-posted the challenge on every
|
status pending"`, because the client re-posted the challenge on every
|
||||||
poll. That was a real renewal bug and is fixed; the 400s are gone and the
|
poll. That was a real renewal bug and is fixed; this run logged no 400 at
|
||||||
client polls as RFC 8555 section 7.5.1 says to.
|
all, and the client polls as RFC 8555 section 7.5.1 says to.
|
||||||
|
|
||||||
The suite still doesn't pass here: pebble never validates the TLS-ALPN
|
The suite still doesn't pass here: pebble never validates the TLS-ALPN
|
||||||
challenge, so the authorizations stay pending until the client gives up
|
challenge, so the authorizations stay pending until the client gives up
|
||||||
and no certificate is issued. Validation needs pebble, in its container,
|
and no certificate is issued — the failure is an `Option::unwrap()` on
|
||||||
to reach the test server's `0.0.0.0:8899` across the docker bridge, and
|
the certificate that never arrived (`tests/src/automation/acme.rs:223`).
|
||||||
|
Validation needs pebble, in its container, to reach the test server's
|
||||||
|
`0.0.0.0:8899` across the docker bridge, and
|
||||||
`ufw` is active on this machine. That wasn't proved — standing up a
|
`ufw` is active on this machine. That wasn't proved — standing up a
|
||||||
listener to test it needs a permission this session didn't have — so
|
listener to test it needs a permission this session didn't have — so
|
||||||
before reading anything into an ACME failure, check that path first.
|
before reading anything into an ACME failure, check that path first.
|
||||||
|
|||||||
Reference in New Issue
Block a user