jcoffey-dev
1518c69033
A data store with readReplicas becomes a replicated store. Writes, operator-written SQL and everything outside a read scope go to the primary. JMAP reads before a request's first write, IMAP LIST, STATUS, SEARCH, SORT and FETCH, POP3 RETR and TOP, DAV GET, PROPFIND and REPORT, and blob downloads run in a read scope. Only account data (properties, indexes, change logs, counters, ACLs, blobs, the search index) is read from a replica; the registry, in-memory values, the task queue and the rest stay on the primary. In a scope, the first read picks a replica round-robin among those up and under the lag limit, and only if it has every change this node has written or heard of for the scope's accounts: marks come from write results, the cluster's state-change broadcasts, a sinceState the client presents, and, with more than one node, Redis. A write inside the scope sends the rest of it to the primary. A miss on a replica is looked up on the primary, and a replica error retries the read there and marks the replica down. Each node samples lag every second (WAL positions on PostgreSQL; GTID sets or Seconds_Behind_Source on MySQL), stops reading from a replica over 5 s and starts again under 2.5 s, and probes a down replica every 10 s. At startup a replica is left out if it's the primary, isn't read-only, applies out of commit order, or doesn't show a marker written to the primary within six tries. replica_tests (postgres, STORE=PostgreSqlReplicated) runs a primary and a streaming hot standby in containers: tests 9, 10, 12, 13, 14 and 15 pass.
90 lines
2.8 KiB
Rust
90 lines
2.8 KiB
Rust
/*
|
|
* SPDX-FileCopyrightText: 2026 Coffey Labs
|
|
*
|
|
* SPDX-License-Identifier: AGPL-3.0-only
|
|
*/
|
|
|
|
//! Scale-out storage (`docs/spec/features/scale-out-storage.md`): sharded
|
|
//! blob stores (ST-16 to ST-22) and sharded in-memory and lookup stores
|
|
//! (ST-23 to ST-29). Each is one more variant of the store enums, whose
|
|
//! members are ordinary stores.
|
|
|
|
pub mod blob;
|
|
pub mod layout;
|
|
pub mod memory;
|
|
pub mod replica;
|
|
pub mod replica_health;
|
|
|
|
/// Calls a PostgreSQL or MySQL backend directly. Replicated stores use it
|
|
/// instead of going back through `Store`, whose futures would then contain
|
|
/// themselves.
|
|
#[macro_export]
|
|
#[doc(hidden)]
|
|
macro_rules! sql_backend {
|
|
($store:expr, $backend:ident => $call:expr) => {
|
|
match $store {
|
|
#[cfg(feature = "postgres")]
|
|
$crate::Store::PostgreSQL($backend) => $call,
|
|
#[cfg(feature = "mysql")]
|
|
$crate::Store::MySQL($backend) => $call,
|
|
_ => Err(trc::StoreEvent::NotConfigured
|
|
.into_err()
|
|
.details("A replicated store's member isn't PostgreSQL or MySQL")),
|
|
}
|
|
};
|
|
}
|
|
|
|
pub use blob::ShardedBlobStore;
|
|
pub use memory::ShardedInMemoryStore;
|
|
|
|
/// A key's home: `xxh3_64(key) mod N`, seed 0, over the whole key (ST-16).
|
|
/// Fixed forever once shipped.
|
|
pub fn home(key: &[u8], members: usize) -> usize {
|
|
(xxhash_rust::xxh3::xxh3_64(key) % members.max(1) as u64) as usize
|
|
}
|
|
|
|
/// A URL without its user information, for records and logs.
|
|
pub fn without_credentials(url: &str) -> String {
|
|
match url.split_once("://") {
|
|
Some((scheme, rest)) => {
|
|
let rest = match rest.split_once('/') {
|
|
Some((authority, path)) => {
|
|
let host = authority.rsplit_once('@').map_or(authority, |(_, h)| h);
|
|
format!("{host}/{path}")
|
|
}
|
|
None => rest.rsplit_once('@').map_or(rest, |(_, h)| h).to_string(),
|
|
};
|
|
format!("{scheme}://{rest}")
|
|
}
|
|
None => url.rsplit_once('@').map_or(url, |(_, h)| h).to_string(),
|
|
}
|
|
}
|
|
|
|
#[cfg(test)]
|
|
mod tests {
|
|
use super::*;
|
|
|
|
#[test]
|
|
fn places_and_hides_credentials() {
|
|
// Stable placement: these values must never change (ST-16)
|
|
assert_eq!(home(b"", 3), (xxhash_rust::xxh3::xxh3_64(b"") % 3) as usize);
|
|
let spread = (0u32..3000)
|
|
.map(|n| home(&n.to_be_bytes(), 3))
|
|
.fold([0; 3], |mut acc, h| {
|
|
acc[h] += 1;
|
|
acc
|
|
});
|
|
assert!(spread.iter().all(|n| *n > 800), "{spread:?}");
|
|
|
|
assert_eq!(
|
|
without_credentials("redis://user:secret@host:6379/0"),
|
|
"redis://host:6379/0"
|
|
);
|
|
assert_eq!(without_credentials("rediss://:pw@host"), "rediss://host");
|
|
assert_eq!(
|
|
without_credentials("redis://host:6379"),
|
|
"redis://host:6379"
|
|
);
|
|
}
|
|
}
|