From 7c27bae4f348fec655ece13cefb75a5d03ee0952 Mon Sep 17 00:00:00 2001 From: John Coffey Date: Sat, 19 Sep 2026 22:41:59 -0700 Subject: [PATCH] Cutover: ext4, so copy in two passes and time the second one The mail host is ext4 (John, 2026-09-19). There is no filesystem snapshot to take, so the sequence as written puts the whole store inside the downtime: stop, copy everything, start. An rsync before the stop and a second one after it moves the bulk while mail is still flowing and leaves only the delta in the window. The first pass is knowingly inconsistent and exists only as a warm-up; the second, once the process has actually exited, is what makes the copy consistent. A RocksDB store suits this, being mostly immutable SST files: what changes between the passes is the WAL, the MANIFEST and any compaction output. Step 3 now says so, and says to time both during the rehearsal, because the second pass is the window and nobody knows yet how long it is. --- docs/spec/cutover.md | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/docs/spec/cutover.md b/docs/spec/cutover.md index 0aef301..37c4527 100644 --- a/docs/spec/cutover.md +++ b/docs/spec/cutover.md @@ -88,6 +88,27 @@ unprivileged process hold port 25. the original is the rollback. Budget the disk for two full copies, and take it from the stopped server, never from under a running one. + **Copy it in two passes, or the window is the whole store.** The host is + ext4, so there is no filesystem snapshot to take and the naive sequence + makes every byte part of the downtime. Instead: + + ```sh + rsync -aH --delete /opt/stalwart/data/ /var/lib/inbuxa/ # server up + systemctl stop stalwart && systemctl disable stalwart # the window opens + rsync -aH --delete /opt/stalwart/data/ /var/lib/inbuxa/ # only the delta + ``` + + The first pass is deliberately inconsistent and that is fine: it exists + only to move the bulk while mail is still flowing. The second pass, after + the server has fully exited, is what makes the copy consistent, and it + moves very little — a RocksDB store is mostly immutable SST files, so + what changes between the passes is the WAL, the MANIFEST and whatever + compaction has just written. Wait for the process to exit, not just for + `systemctl stop` to return. + + Time both passes during the rehearsal. The second one is the number that + matters: it is the window. + **Then make the original read-only, before the fork exists on this host** (`chmod -R a-w`, or `chattr +i` on the directory, or keep it on a read-only bind mount). Until this moment the running server's own RocksDB