Capture and surface the settings migrate_v016.py does NOT migrate

Ran the converter against a real production settings corpus pulled from the
instance this tool is being built to migrate (secrets scrubbed on that host;
nothing sensitive transited). The result reframes what "a successful
migration" means:

    total settings:     12401
    NOT migrated:       12182  (98.2%)
    actually migrated:  219  (1.8%)

Stalwart's own script reports this in an unmigrated.txt it writes beside its
output - and this tool was throwing that file away. Worse, RunSettingsConvert
never set cmd.Dir, so the script wrote unmigrated.txt into whatever directory
the operator happened to launch from, or failed the whole convert when that
directory wasn't writable. Both reproduced.

The largest groups left behind on production are spam-filter rules, DNSBLs,
trusted-domain and URL-redirector lookups, queue scheduling and TLS
settings - and server.listener. That last one explains something that had
been puzzling from an earlier smoke run: a freshly migrated 0.16 instance
answered on none of the ports the old one did, and served nothing but
/admin. Its listeners never migrated.

So:

- SettingsConvertOptions gains WorkDir, and the convert runs there. The
  report lands somewhere known and an unwritable cwd can't fail the step.
- ReadUnmigratedReport parses it; UnmigratedReport.Summary renders the
  largest groups first.
- The dry run records it as an "unmigrated-settings" artifact with a
  checksum, notes the count in the checkpoint step, and prints it as a
  standing warning rather than a footnote.

ARCHITECTURE 4.3 anticipated a "best-effort apply-plan" gap here. The gap is
not a few stragglers needing review; it is effectively the entire
configuration, and the tool has to say so where nobody can miss it.

Verified against the smoke VM: even a default `stalwart --init` instance
reports 3505 of ~3514 settings unmigrated, listeners included.
This commit is contained in:
2026-08-23 20:00:23 -07:00
parent 680e554f23
commit b88724632c
5 changed files with 220 additions and 9 deletions
+28 -1
View File
@@ -196,21 +196,48 @@ func runRun(args []string) (err error) {
}
fmt.Printf("cloned verified backup into sandbox: %s\n", sandboxDataDir)
unmigratedPath := filepath.Join(runWorkDir, "unmigrated.txt")
if _, err := store.RunStep(rs, checkpoint.PhaseStage, "convert-settings", func() (checkpoint.StepOutcome, error) {
if err := backup.RunSettingsConvert(ctx, backup.SettingsConvertOptions{
PythonPath: *pythonPath, ScriptPath: scriptDest,
SettingsPath: settingsPath, PrincipalsPath: principalsPath,
ConfigPath: sandboxConfigPath, OutputPath: sandboxExportPath,
PatchPaths: map[string]string{*dataDir: sandboxDataDir},
// Without this the script writes unmigrated.txt into whatever
// directory this command was launched from - or fails outright
// if that isn't writable.
WorkDir: runWorkDir,
}); err != nil {
return checkpoint.StepOutcome{}, err
}
return checkpoint.StepOutcome{Detail: fmt.Sprintf("generated %s and %s, patched to point at the sandbox", sandboxConfigPath, sandboxExportPath)}, nil
detail := fmt.Sprintf("generated %s and %s, patched to point at the sandbox", sandboxConfigPath, sandboxExportPath)
if report, err := backup.ReadUnmigratedReport(unmigratedPath); err == nil && report != nil && report.TotalKeys > 0 {
detail += fmt.Sprintf("; %d setting(s) were NOT migrated", report.TotalKeys)
}
return checkpoint.StepOutcome{Detail: detail, Extra: unmigratedPath}, nil
}); err != nil {
return fmt.Errorf("convert settings: %w", err)
}
fmt.Println("generated sandbox config.json and export.json")
// What the converter could NOT carry over matters more than what it
// could: against a real instance this is the overwhelming majority of
// the configuration, including the listeners, and an operator who
// doesn't read it will bring up a server that answers on no ports.
unmigrated, err := backup.ReadUnmigratedReport(unmigratedPath)
if err != nil {
fmt.Fprintf(os.Stderr, "warning: couldn't read the unmigrated-settings report: %v\n", err)
} else if unmigrated != nil && unmigrated.TotalKeys > 0 {
if sum, size, hashErr := backup.HashFile(unmigratedPath); hashErr == nil {
rs.RecordArtifact("unmigrated-settings", checkpoint.Artifact{Path: unmigratedPath, SHA256: sum, SizeBytes: size})
if saveErr := store.Save(rs); saveErr != nil {
fmt.Fprintf(os.Stderr, "warning: couldn't record the unmigrated-settings artifact: %v\n", saveErr)
}
}
fmt.Printf("\n !! %s\n", unmigrated.Summary(10))
fmt.Println(" These do not carry over. Recreate them on the migrated instance before it serves mail.")
}
fmt.Println("\n--- recovery-mode migration (against the sandbox) ---")
listenURL := fmt.Sprintf("http://127.0.0.1:%d/", *recoveryPort)
recReport, err := recovery.Run(ctx, store, rs, recovery.Options{