main
11
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 jcoffey-dev
|
12ec0c3fd4 |
Rename the module to the Coffey-Labs organisation
The repositories moved off LINUXexpert-org. Here that is not a documentation change: the old organisation was the module path, so it is declared in go.mod and repeated in every internal import. Leaving it would have been worse than a stale link. GitHub redirects the repository, but a go.mod whose module line disagrees with the path it was fetched from is an error rather than a redirect, so `go get` on the new address would have failed against the old declaration. go.mod, 34 files of imports, and the repository links in README and ARCHITECTURE. go mod tidy leaves go.sum untouched -- no dependency moved, only our own path. |
||
|
jcoffey-dev
|
77e80cb9f2 |
Change the copyright holder to Coffey Labs
119 SPDX-FileCopyrightText headers and the README's licence line. The distinction that matters here: LINUXexpert-org appears in this repository in two completely different roles. As a copyright holder in the SPDX headers, which is what changes, and as the GitHub organisation in the module path and 64 import statements, which does not -- the repository still lives at github.com/LINUXexpert-org/stalwart-migrator, and rewriting that would not be a licence change, it would break the build. Both replacements are anchored to their copyright forms, so an import path cannot match either. Import count is 64 before and after, and go.mod is untouched. LICENSE untouched: the FSF's copyright on the GPL text and the "<name of author>" placeholders are not ours to edit. go vet, go build and go test all clean. |
||
|
jcoffey-dev
|
edd8279743 |
Refuse a Docker deployment before stopping anything
Cutover already refused a container -- recreating one from a new image is not swapping a binary and rewriting a unit, and this tool does not automate it -- but it refused from cutover.Run, which run.go calls at line 324. The service is stopped at line 243. So the sequence on a container was: stop Stalwart, convert the settings, then discover the deployment cannot be cut over, return the error, and exit with mail still down. Reported as #1. The deployment kind is known in preflight, before anything has been touched, and that is now where it is acted on: docker is a blocking check. rehearse keeps working -- it never stops the service or cuts over, and telling an operator what the migration involves is most useful precisely when the tool cannot do it for them -- so it sets DeploymentCheckAdvisory, alongside the ToolCheckAdvisory it already set for the same reason. The second half is not docker's alone. Every return between the stop and the end of cutover returned with the service down; a failed settings conversion would have done the same to a systemd host. run now registers a restart on the way out, after the stop rather than before, so it only ever starts something this tool stopped. It does not claim to have recovered the migration -- a part-migrated store still needs --resume or the operator's recovery point -- it removes the narrower failure of exiting on a foreseeable error while the server it stopped stays stopped. |
||
|
jcoffey-dev
|
0739866c14 |
Let a host with no internet supply the migration script itself
`run` and `rehearse` always fetched migrate_v016.py from GitHub, so a mail server with no route out could not be migrated at all - an air-gapped host, or a clone deliberately cut off so it cannot renew certificates or deliver queued mail for the domains it was copied from. --migration-script takes a local copy instead, still checked against --migration-script-sha256 when one is pinned. Found while staging a production clone for a dress rehearsal: the clone has no route out on purpose, and that is exactly the property that stops a copy of a live mail server doing something in the real world. |
||
|
jcoffey-dev
|
e955a41d58 |
Fix the domain/tenant mismatch that failed the second live migration
The second production attempt failed during recovery-mode apply, with the
mail server already stopped and the store already at schema v6:
create Account restore-13: invalidForeignKey | Object id: Domain#d
v0.16 requires a tenant-scoped Account to sit on a Domain owned by that
same tenant, for its primary domain and for every alias. v0.15 imposed no
such rule, and migrate_v016.py carries the two facts over independently:
_build_domains sets a domain's memberTenantId only for domains declared as
their own `domain` principal with a `tenant`, while _build_user sets the
account's from the account's own record. A domain that exists only inside
an email address is inferred, gets no tenant, and every tenant-scoped
account using it is then rejected.
Established by reproduction rather than inference: a synthetic v0.15
principal dump, run through the unpatched upstream converter and applied to
a real 0.16.14 in recovery mode, reproduces the error character for
character - the `#d` is the server's own object id for the offending
domain, not a plan client-id. The same harness establishes which directions
are constrained: a tenant-scoped account on a tenant-less domain or on
another tenant's domain is rejected; a global account on a tenant-owned
domain is accepted.
- applyplan.ReconcileDomainTenants repairs the plan between convert and
apply. Where a tenant-less domain is used only by accounts of one
tenant, the domain adopts that tenant - the sole assignment that both
applies and keeps every account. Where accounts genuinely disagree it
changes nothing and reports why, because forcing such a plan through
would mean dropping mailboxes.
- stalwartapi.FetchTenantLayout maps tenant membership over the 0.15 REST
API and predicts the outcome with the same rule the server enforces, so
preflight either warns about the domains that will adopt a tenant or
fails - while the service is still running.
- The plan is parsed generically rather than through the typed Operation.
A real export.json mixes shapes: `create` maps a client-id to an object,
`update` carries a flat one. The typed form failed on the first `update`
line, found by running against actual converter output. Numbers decode
as json.Number so a 10 GiB quota is not rewritten as 1.073741824e+10.
Corrects the record: the previous commit claimed the converter emits every
Account with `tenantId: null` and made preflight refuse every multi-tenant
install on that basis. The field is memberTenantId, the converter does
populate it, and the export had been inspected for a key no version of the
script ever writes. The refusal is now narrowed to what v0.16 genuinely
cannot represent.
The same fix has been prepared for migrate_v016.py upstream. The tool
downloads that script rather than vendoring it, so the repair stays here
until a released version carries it, and is a no-op on a consistent plan.
|
||
|
jcoffey-dev
|
c29140b6b3 |
Refuse multi-tenant installs in preflight, not after the service is stopped
A second live attempt failed in the same shape as the first: preflight
clean, settings dumped and converted, then a failure during recovery-mode
migration with the mail server already stopped.
created Tenant (1)
created Domain (9)
create Account restore-13: invalidForeignKey | Object id: Domain#d
migrate_v016.py carries the Tenant and the Domains but emits every Account
with a null tenantId, so the account references a tenant-owned domain while
belonging to no tenant and the foreign key is rejected. That is Stalwart's
converter and there is nothing this tool can do about it: a multi-tenant
install has to be migrated by hand until the converter handles tenants.
What this tool got wrong was the timing. Tenant principals are one API call
away and were readable the entire time the server was running. Preflight now
queries them and fails before anything is touched, with an explanation of
exactly what would otherwise fail and when.
This is the same lesson as the stalwart-cli check: knowable in advance,
discovered after a production mail server had been stopped, twice. Any
dependency of the conversion belongs in preflight, not in the phase that
consumes it.
Also makes the external-tool checks advisory during `rehearse`, which never
invokes stalwart-cli - refusing to run read-only reconnaissance because the
operator lacks a tool that reconnaissance would tell them to get was
backwards.
|
||
|
jcoffey-dev
|
9faa21f4f1 |
Fix the three defects that cost a production restore
A live migration on 2026-08-24 stopped a production mail server and then
discovered the host's stalwart-cli was 0.13.4 - present, but from when the
CLI shipped with the server, with no `apply` command. The migration needs
v1.0.2+ from the separately-versioned stalwartlabs/cli repository.
Recovery was closed in both directions. v0.16's recovery-mode boot had
already bumped the store schema to v6, so the 0.15.5 binary refused to
reopen it ("expected 5 or below, found 6"). Going forward needed
export.json, which this tool's own failure path had deleted - and
regenerating it required a settings dump from a live v0.15 instance that
could no longer start. The operator restored a day-old snapshot and lost a
day of mail across nine domains.
Three fixes:
1. preflight.CheckExternalTools verifies stalwart-cli exists and is v1.0.2
or later, and that python3 runs - before anything is touched. Every fact
needed to prevent this was available in under a second from a stopped
state. Skipped for a patch upgrade, which invokes neither tool.
2. A failed run no longer deletes its work directory. Cleaning up on every
exit path was right for a sandboxed rehearsal and catastrophic here:
once the service is stopped the settings dump cannot be regenerated, so
deleting it removes the only way forward. The failure now prints the
resume command instead.
3. `run --resume <id>` continues an interrupted run. The checkpoint
machinery existed but never engaged, because run created a new run every
invocation - so a retry re-ran preflight against a binary already moved
aside, and failed. Completed steps are skipped from the checkpoint.
Proven against a VM built to match the failure: stalwart-cli 0.15.5,
accounts and mail seeded.
* preflight refused, service still active, mail still accepted
* a stub CLI passing --version and failing apply left the run stopped
with all eight inputs intact and the resume command printed
* --resume carried it to a clean finish: five seconds of downtime,
listeners regenerated, admin role restored, quotas rebuilt
That failure-path test is the one that should have run before production.
Every earlier test had stalwart-cli installed from the start, and the one
failure I did exercise happened to leave its artifacts behind.
|
||
|
jcoffey-dev
|
0d83283caa |
Classify the unmigrated settings instead of just counting them
A production rehearsal reported 12,182 settings not carried over by
migrate_v016.py. As a bare number that reads as an impossible amount of
manual reconstruction, and it is misleading. Snapshotting a v0.16.14 store
migrated from a real v0.15.5 showed what those settings actually are:
8547 regenerates server.blocked-ip
3337 shipped with v0.16 lookup.url-redirectors, lookup.trusted-domains,
spam-filter.list, spam-filter.rule,
spam-filter.dnsbl, lookup.surbl-hashbl
224 already carried server.listener, signature.* (DKIM)
293 NEED YOUR REVIEW queue.schedule, config.local-keys,
server.auto-ban, spam-filter.llm, queue.tls, ...
server.blocked-ip is auto-ban state that repopulates from live traffic. The
stock groups are data v0.16 provides itself - 2,084 MemoryLookupKey, 66
SpamRule and 18 SpamDnsblServer objects were already present in the migrated
store. DKIM came across as DkimSignature objects with private keys intact,
verified on that instance. So the real worklist is ~293 keys, not 12,182.
backup.UnmigratedReport.Classify encodes this and the rehearsal now reports
the categorised view. Rules match longest-prefix-first, because
server.blocked-ip is runtime state while server.auto-ban beside it is
configuration, and an unrecognized prefix defaults to "needs review" -
assuming an unknown setting is safe to ignore is the wrong default.
This also retired the lookup and spam-filter generators that were the
planned next step. v0.15's rules are stwt_rbl_senderscore_ip; v0.16's are
STWT_RBL_SENDERSCORE_IP - the same stock set, already installed. Generating
them from v0.15 would duplicate every rule and revert upstream updates, so
they were deliberately not written. The targets worth generating are the
small site-specific groups instead: queue.schedule, queue.tls,
session.auth, server.auto-ban.
No production data in this commit: the test fixture uses the real group
names and counts with example.com standing in for customer domains.
|
||
|
jcoffey-dev
|
28c0fa57cb |
Restore administrator roles that migrate_v016.py drops
Chased down why a migrated instance had no working administrator. The
account authenticated fine and was refused every management call, and the
cause is that migrate_v016.py assigns every migrated account the User role
regardless of what it held before: an account that was `roles: ["admin"]`
in v0.15 comes out the far side as `roles: {"@type": "User"}`.
Ordinary users were never affected - User is what they had and what they
get - and their credentials, mail and mailboxes survive untouched. It is
specifically administrators who lose their privileges, which is a bad thing
to discover after cutting over.
The v0.16 shape came from the server's own schema document rather than the
published reference: GET /api/schema defines x:UserRoles as a multi-variant
type with variants User, Admin and Custom. Account is itself multi-variant,
so an upsert needs its own "@type" too - without it the server rejects the
operation outright ("upsert entry is missing `@type`").
applyplan.AccountRoleOperations restores roles from the principals dump,
emitting operations only for accounts whose role actually changes.
Rewriting every account would be a much larger blast radius for no benefit.
Where v0.15 listed several roles, admin wins - under-privileging an
administrator locks them out, which is the failure being fixed - and the
collapse is reported rather than done silently, as are roles with no known
v0.16 equivalent.
Verified end to end on the smoke VM: rehearse against the real 0.15.5 put
the role operation in the supplement, applying that supplement to a
migrated 0.16.14 whose admin was broken restored management access
(accounts=3), and alice and bob logged in over IMAPS with unchanged
credentials, read their mail, and accepted new SMTP delivery.
Also recorded: x:Account.domainId returns an internal id on v0.16, not a
domain name, so the post-migration directory comparison would read every
domain as missing. Resolving that needs an x:Domain/get call not yet
confirmed against the binary.
|
||
|
jcoffey-dev
|
3bd694114f |
Generate a v0.16 apply plan for the listeners migrate_v016.py leaves behind
First piece of ARCHITECTURE.md 4.3's apply-plan, and the piece that decides
whether a migrated server answers at all: server.listener is not among the
settings the official converter carries, so a freshly migrated instance
binds nothing. Every other unmigrated setting degrades the server; this one
stops it being a server.
internal/applyplan maps server.listener.* onto x:NetworkListener objects and
reports its own coverage. Against the smoke instance that is 24 of 3,505
unmigrated keys - 0.7% - and the output says 0.7%, listing the largest
groups it did not touch. A plan covering a fraction while implying
completeness would be worse than no plan.
The wire format was confirmed against the binary, not the documentation.
The published schema reference gives NetworkListener.bind as a JSON array;
0.16.14 rejects that outright ("Invalid value for object property.
Properties: bind"). The encoding it accepts is a value-keyed set,
{"[::]:25": true}, found by applying a plan to a live recovery-mode 0.16.14
and reading it back with `stalwart-cli snapshot`. Only mappings confirmed
that way are in DefaultGenerators; managesieve -> manageSieve is the one
protocol whose spelling changes, and an unrecognized protocol is reported
and skipped rather than passed through to fail at apply time.
Operations are upserts matched on name, so a plan can be re-run - an
operator will run it more than once - and the supplement is applied after
export.json rather than merged into it, so a generated mapping can never
override one the official script got right.
Verified end to end: rehearse against a real 0.15.5 generated ten
listeners, `stalwart-cli apply` created all ten on a real 0.16.14 with zero
failures, a snapshot read them back with correct protocols, binds and TLS
flags, and re-applying reported 10 updated / 0 created / 0 failed.
|
||
|
jcoffey-dev
|
a0f846a31b |
Replace the sandbox dry run with a read-only rehearsal
`run --dry-run` cloned the data directory into a sandbox, migrated the copy,
booted it, and compared content before and after. Running that design
against a real 0.15.5 instance and a real production settings corpus
retired it:
* The mechanics were never the risk. Backup, dump, convert and the
recovery-mode store migration all worked essentially first time.
* Its final comparison cannot work at all. It needs the migrated sandbox
to answer an API, and server.listener is not among the settings
migrate_v016.py carries - so a migrated instance has no listeners and
answers on nothing. That is the true post-migration state, not a
sandbox artifact to engineer around.
* The expensive half bought the least: against a 3.6 GB production store
it copies the data twice, reading a live mail store, to prove RocksDB
files copy and recovery mode can open them.
Meanwhile the cheap half found every problem that would have derailed a
real migration - an empty defaultHostname v0.16 rejects, passwords v0.16
refuses to create, and a 12,182-key reconstruction worklist - and needs no
data copy at all.
So `stalwart-migrate rehearse`: preflight, dump, convert, report. It copies
nothing, starts no server, and never writes to the store, so it is safe to
run against production repeatedly without a maintenance window. It needs no
target binary either, since convert is pure Python.
The scratch directory is cleaned up as before, with the rehearsal's two
conclusions lifted out first and recorded as artifacts: export.json (what
will carry over) and unmigrated.txt (what will not). Recording an artifact
whose path was about to be deleted was a bug in the first cut of this;
both now resolve.
`run` keeps its refusal and explains where rehearse went. `--dry-run` is
kept as a flag purely to say what replaced it.
Verified against the smoke VM end to end: rehearsal completes read-only in
seconds and reports 3505 unmigrated settings on a default install,
listeners included.
|