Add local login, agent extra log paths, IPv4/IPv6 metrics; remediate security audit findings

This is a large squashed commit covering two batches of prior uncommitted
work plus a full security-audit remediation pass, kept together because
go.mod/go.sum and several shared files (main.go, handler.go) were touched
by both and splitting risked non-building intermediate commits.

Features (built earlier, previously uncommitted):
- Local username/password login for single-tenant deployments with no
  SSO configured (api/localauth, alerting/internal/sessioncheck,
  sentryctl users, web/src/routes/login, metadata migrations 0040/0041).
- Remotely-editable additional log file paths for agents, on top of
  their existing primary source (api/agents, agent/sentry-agent
  extra-file-path diffing, web agent config UI).
- IPv4/IPv6 addresses reported alongside other host system metrics.

Security audit remediation (this pass, all live-verified in production):
- Critical: block ClickHouse SSRF table functions (url/remote/file/s3/...)
  in the raw-SQL query escape hatch.
- High: deny sensitive paths and require Admin to add agent
  extra_file_paths (Editor could previously point an agent at /etc/shadow
  or an SSH key); alerting webhook targets now validate against
  internal/metadata/loopback addresses, both at creation and send time;
  alerting's session middleware now enforces an Editor+ floor on
  mutating requests instead of "any authenticated session"; bumped
  goxmldsig to close a SAML signature-verification bypass (GO-2026-4753).
- Medium: per-IP login rate limiting; security response headers
  (HSTS/CSP/nosniff/X-Frame-Options/Referrer-Policy/Permissions-Policy)
  on web/nginx.conf; a DevCredentialWarnings check in every Go service's
  config loader, logging loudly at startup if a deployment is still on
  docker-compose.yml's literal dev-only credentials; dependency bumps
  (golang.org/x/text, grpc, x/net, quick-xml, h2) across every affected
  Go module and both Rust crates, including a previously-uncovered x/net
  vulnerability in deploy/operator; a new security-scan.yml CI workflow
  running cargo-deny/govulncheck/npm-audit, mirroring the existing
  license-compliance.yml matrix shape.
- Low: removed sentryctl's plaintext --password flag (shell
  history/`ps` exposure) in favor of stdin and a --password-stdin flag
  for reset-password's optional specific-password path; a dummy bcrypt
  comparison closes a login response-time username-enumeration
  side-channel.
This commit is contained in:
2026-08-18 23:53:20 -07:00
parent d2bb9de245
commit 4b5dae5879
87 changed files with 5095 additions and 164 deletions
+8 -8
View File
@@ -258,7 +258,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "39cab71617ae0d63f51a36d69f866391735b51691dbda63cf6f96d042b63efeb"
dependencies = [
"libc",
"windows-sys 0.61.2",
"windows-sys 0.52.0",
]
[[package]]
@@ -348,9 +348,9 @@ dependencies = [
[[package]]
name = "h2"
version = "0.4.15"
version = "0.4.16"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6cb093c84e8bd9b188d4c4a8cb6579fc016968d14c99882163cd3ff402a4f155"
checksum = "a9f37a958b41b3b19ee2707c06439c0e9e547e847223eb791ecb0cb821c65e27"
dependencies = [
"atomic-waker",
"bytes",
@@ -477,7 +477,7 @@ dependencies = [
"hyper",
"libc",
"pin-project-lite",
"socket2 0.6.5",
"socket2 0.5.10",
"tokio",
"tower-service",
"tracing",
@@ -737,9 +737,9 @@ dependencies = [
[[package]]
name = "quick-xml"
version = "0.36.2"
version = "0.41.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f7649a7b4df05aed9ea7ec6f628c67c9953a43869b8bc50929569b2999d443fe"
checksum = "e660451e55124f798a69a5af3f49ccfbefbd41910eefd25caf2393e1f3473ec1"
dependencies = [
"memchr",
]
@@ -842,7 +842,7 @@ dependencies = [
"errno",
"libc",
"linux-raw-sys",
"windows-sys 0.61.2",
"windows-sys 0.52.0",
]
[[package]]
@@ -1079,7 +1079,7 @@ dependencies = [
"getrandom 0.4.3",
"once_cell",
"rustix",
"windows-sys 0.61.2",
"windows-sys 0.52.0",
]
[[package]]
+6 -1
View File
@@ -8,8 +8,13 @@ WORKDIR /src
COPY proto ./proto
COPY agent ./agent
WORKDIR /src/agent
# file-tail is additive (no extra deps -- see Cargo.toml's [features]
# doc comment) and needed on any host whose logs aren't in journald
# (e.g. nginx's plain access/error log files), so it's built in
# alongside the journald default rather than left as an opt-in most
# deployments would otherwise have to remember to ask for.
RUN rustup target add x86_64-unknown-linux-musl \
&& cargo build --release --target x86_64-unknown-linux-musl -p sentry-agent
&& cargo build --release --target x86_64-unknown-linux-musl -p sentry-agent --features journald,file-tail
FROM scratch
COPY --from=builder /src/agent/target/x86_64-unknown-linux-musl/release/sentry-agent /sentry-agent
+24
View File
@@ -153,6 +153,30 @@ alerting engine already detects natively via an `absence`-condition
alert rule. See `/docs/agent-heartbeat-monitoring.md` for the exact rule
to create.
## Host CPU/memory/disk metrics
Same shape as heartbeat, same reasoning: `[metrics]` in the config
(`enabled = false` by default) sends a periodic record — CPU%, memory
used/total, disk used/total for `/` — tagged `sentry.metrics=true`, with
the individual numbers as their own attributes (`cpu_percent`,
`mem_used_bytes`, `mem_total_bytes`, `disk_used_bytes`,
`disk_total_bytes`), queryable directly (e.g. `cpu_percent > 80`) since
the query language transparently maps any non-standard field name to
`attributes['field']` with automatic numeric casting. Powers the web
UI's "Hosts" nav section. Linux-only for now (`src/metrics.rs`) — reads
`/proc/stat`/`/proc/meminfo` and shells out to `df`, no new
dependencies, same "shell out to a boring, ubiquitous tool" precedent
`journalctl` already sets.
**Enable this on only one agent process per physical host.** It's
common for one host to run several `sentry-agent` processes (one per
log source, each needing its own `[agent] host` value to work around
the `agents` table's `UNIQUE (tenant_id, host)` constraint — see
`/docs/agent-management-design.md`) — turning `[metrics]` on for more
than one of them reports the same physical machine as multiple
different "hosts" with conflicting metric series. Pick the one process
using that host's real, unoverridden hostname.
## Running as a Windows service
"A native Windows service, not a WSL wrapper" means implementing the Win32
+1 -1
View File
@@ -47,7 +47,7 @@ windows = { version = "0.58", features = [
"Win32_Security",
] }
windows-service = "0.7"
quick-xml = "0.36"
quick-xml = "0.41"
[build-dependencies]
tonic-build = "0.12"
@@ -51,6 +51,17 @@ interval = "60s"
# interval = "5m"
# interval = "1h"
[metrics]
# Host CPU/memory/disk usage, sent as its own periodic record the same
# way heartbeat is (see web/'s "Hosts" nav section). Off by default --
# unlike heartbeat, this is a deliberate per-host decision: if several
# sentry-agent processes run on the same physical host (e.g. one per log
# source), enable this on only ONE of them, or the same host will report
# multiple conflicting metric series. Linux-only for now. Root disk ("/")
# only -- not configurable in this release.
enabled = false
interval = "60s"
[ingest]
endpoint = "https://ingest.internal:4317"
+50
View File
@@ -15,6 +15,7 @@ pub struct Config {
pub source: SourceConfig,
pub batch: BatchConfig,
pub heartbeat: HeartbeatConfig,
pub metrics: MetricsConfig,
pub ingest: IngestConfig,
pub tls: TlsConfig,
}
@@ -168,6 +169,31 @@ impl Default for HeartbeatConfig {
}
}
/// Off by default -- same "off unless configured" posture as every other
/// optional feature in this codebase -- since collecting host metrics is
/// a deliberate per-host decision (see /agent/README.md's Hosts-feature
/// notes: only one agent process per physical host should have this on,
/// to avoid duplicate/fragmented metric series when several agent
/// processes share a host under different `[agent] host` overrides).
/// Sent independently of `batch`, same reasoning and mechanism as
/// `HeartbeatConfig` above (see main.rs's `send_metrics`).
#[derive(Debug, Clone, Deserialize)]
#[serde(default)]
pub struct MetricsConfig {
pub enabled: bool,
#[serde(deserialize_with = "deserialize_duration")]
pub interval: Duration,
}
impl Default for MetricsConfig {
fn default() -> Self {
Self {
enabled: false,
interval: Duration::from_secs(60),
}
}
}
/// Parses a human-friendly duration string with an explicit unit suffix
/// -- "30s", "5m", "1h" -- deliberately the same s/m/h vocabulary
/// `earliest=`/`latest=` use in the query language
@@ -237,6 +263,30 @@ mod heartbeat_config_tests {
}
}
#[cfg(test)]
mod metrics_config_tests {
use super::*;
#[test]
fn default_is_60_seconds_and_disabled() {
let cfg = MetricsConfig::default();
assert!(!cfg.enabled);
assert_eq!(cfg.interval, Duration::from_secs(60));
}
#[test]
fn toml_field_parses_via_deserialize() {
#[derive(Deserialize)]
struct Wrapper {
#[serde(default)]
metrics: MetricsConfig,
}
let w: Wrapper = toml::from_str("[metrics]\nenabled = true\ninterval = \"30s\"\n").unwrap();
assert!(w.metrics.enabled);
assert_eq!(w.metrics.interval, Duration::from_secs(30));
}
}
#[derive(Debug, Clone, Deserialize)]
#[serde(default)]
pub struct IngestConfig {
+149 -11
View File
@@ -1,6 +1,7 @@
mod batch;
mod config;
mod grpc;
mod metrics;
mod source;
#[cfg(windows)]
@@ -22,6 +23,8 @@ use clap::Parser;
use config::Config;
use pb::agent::v1::{agent_control_client::AgentControlClient, AgentCommand, CheckInRequest, DesiredOverride, ReportedConfig};
use pb::{log_ingest_client::LogIngestClient, LogRecord, Severity};
use std::collections::HashMap;
use std::collections::HashSet;
use std::path::PathBuf;
use std::time::Duration;
use tokio::sync::mpsc;
@@ -92,12 +95,33 @@ pub async fn run_agent(config_path: Option<PathBuf>) -> Result<()> {
let host = cfg.agent.host.clone().unwrap_or_else(default_hostname);
let service = cfg.agent.service.clone();
// One long-lived channel for the whole process, not one per source
// task -- both the primary source and every extra-file-path task
// (see apply_override's extra_file_paths handling) send into clones
// of the same `tx`. This matters specifically because the primary
// source can be aborted and respawned at runtime (a journald_unit
// override): if each respawn created a *new* channel the way this
// used to work, any extra-file-path task still holding a clone of
// the old `tx` would be silently orphaned (sending into a channel
// whose `rx` had just been replaced/dropped). Creating the channel
// once and only ever swapping the *task* (never the channel) avoids
// that entirely.
let (tx, mut rx) = mpsc::channel(1024);
// source_cfg is mutable: a remote DesiredOverride's journald_unit
// field (see apply_override below) can change it at runtime, which
// means aborting and respawning the source task with the new
// filter -- source_handle/rx are mutable for the same reason.
// filter -- source_handle is mutable for the same reason.
let mut source_cfg = cfg.source.clone();
let (mut source_handle, mut rx) = spawn_source_task(source_cfg.clone());
let mut source_handle = spawn_source_task(source_cfg.clone(), tx.clone());
// Extra file paths an operator has remotely added via the web UI
// (see apply_override's extra_file_paths handling) -- empty until
// the first such override arrives. Keyed by path so a later
// override with a different path list can be diffed against what's
// already running: abort tasks for paths that were removed, spawn
// tasks for paths that are new, leave everything else untouched.
let mut extra_file_tasks: HashMap<PathBuf, tokio::task::JoinHandle<()>> = HashMap::new();
let channel = grpc::connect(&cfg.ingest, &cfg.tls)
.await
@@ -116,6 +140,12 @@ pub async fn run_agent(config_path: Option<PathBuf>) -> Result<()> {
let mut flush_interval = Duration::from_millis(cfg.batch.flush_interval_ms);
let mut heartbeat_enabled = cfg.heartbeat.enabled;
let mut heartbeat_interval = cfg.heartbeat.interval;
// Not remotely overridable (unlike batch/heartbeat above) -- see
// /agent/README.md's Hosts-feature notes on why this is a
// deliberate, per-host, config-file-only decision, not something
// the web UI can flip on for an arbitrary agent.
let metrics_enabled = cfg.metrics.enabled;
let metrics_interval = cfg.metrics.interval;
// Empty until the first override is ever applied -- echoed back on
// every CheckIn as-is so the server can tell "pending" (an edit
// exists this agent hasn't picked up) from "applied."
@@ -134,8 +164,17 @@ pub async fn run_agent(config_path: Option<PathBuf>) -> Result<()> {
// heartbeat_enabled.
let mut heartbeat_ticker = tokio::time::interval(heartbeat_interval.max(Duration::from_millis(50)));
// Unlike heartbeat_ticker, metrics has no secondary purpose (no
// CheckIn-equivalent side effect) to keep it running when disabled,
// so the select! arm below skips it entirely via `if metrics_enabled`
// rather than always firing and conditionally acting.
let mut metrics_ticker = tokio::time::interval(metrics_interval.max(Duration::from_millis(50)));
loop {
tokio::select! {
_ = metrics_ticker.tick(), if metrics_enabled => {
send_metrics(&mut client, &host, &service).await;
}
_ = heartbeat_ticker.tick() => {
if heartbeat_enabled {
send_heartbeat(&mut client, &host, &service).await;
@@ -164,7 +203,8 @@ pub async fn run_agent(config_path: Option<PathBuf>) -> Result<()> {
&mut batch_max_size, &mut flush_interval,
&mut heartbeat_enabled, &mut heartbeat_interval,
&mut batcher, &mut ticker, &mut heartbeat_ticker,
&mut source_cfg, &mut source_handle, &mut rx,
&mut source_cfg, &mut source_handle,
&mut extra_file_tasks, &tx,
&mut client,
).await;
applied_override_version = ov.version.clone();
@@ -272,7 +312,8 @@ async fn apply_override(
heartbeat_ticker: &mut tokio::time::Interval,
source_cfg: &mut config::SourceConfig,
source_handle: &mut tokio::task::JoinHandle<()>,
rx: &mut mpsc::Receiver<source::RawLine>,
extra_file_tasks: &mut HashMap<PathBuf, tokio::task::JoinHandle<()>>,
tx: &source::LineSender,
client: &mut LogIngestClient<Channel>,
) {
if let Some(v) = ov.batch_max_size {
@@ -310,19 +351,57 @@ async fn apply_override(
if *current_unit != new_unit {
*source_cfg = config::SourceConfig::Journald { unit: new_unit };
source_handle.abort();
let (new_handle, new_rx) = spawn_source_task(source_cfg.clone());
*source_handle = new_handle;
*rx = new_rx;
*source_handle = spawn_source_task(source_cfg.clone(), tx.clone());
tracing::info!(unit = ?unit, "applied remote journald unit override, restarted source");
}
}
}
// Extra file paths to tail alongside whatever the primary source
// above is -- see agent_control.proto's DesiredOverride.
// extra_file_paths comment. Diffed against what's already running
// (extra_file_tasks' keys) rather than blindly tearing everything
// down and respawning: an edit to, say, heartbeat_interval_ms must
// never interrupt a file that's already being tailed and hasn't
// changed. `ov.extra_file_paths` is the complete desired list every
// time (never a partial patch -- see the proto field's own doc
// comment), so anything not in it gets removed.
let desired: HashSet<PathBuf> = ov.extra_file_paths.iter().map(PathBuf::from).collect();
let current: HashSet<PathBuf> = extra_file_tasks.keys().cloned().collect();
for removed in current.difference(&desired) {
if let Some(handle) = extra_file_tasks.remove(removed) {
handle.abort();
tracing::info!(path = %removed.display(), "stopped tailing removed extra file path");
}
}
for added in desired.difference(&current) {
extra_file_tasks.insert(added.clone(), spawn_extra_file_task(added.clone(), tx.clone()));
tracing::info!(path = %added.display(), "started tailing new extra file path");
}
}
fn spawn_source_task(source_cfg: config::SourceConfig) -> (tokio::task::JoinHandle<()>, mpsc::Receiver<source::RawLine>) {
let (tx, rx) = mpsc::channel(1024);
let handle = tokio::spawn(spawn_source(source_cfg, tx));
(handle, rx)
fn spawn_source_task(source_cfg: config::SourceConfig, tx: source::LineSender) -> tokio::task::JoinHandle<()> {
tokio::spawn(spawn_source(source_cfg, tx))
}
/// Same shape as `spawn_source`'s own per-source-kind feature gating --
/// an extra file path is really just another `File` source, tailed
/// from the end (no `from_beginning` knob for these; matches the
/// primary `file` source's own sensible default), feeding into the
/// same shared channel every other source in this process uses.
#[cfg_attr(not(feature = "file-tail"), allow(unused_variables))]
fn spawn_extra_file_task(path: PathBuf, tx: source::LineSender) -> tokio::task::JoinHandle<()> {
tokio::spawn(async move {
#[cfg(feature = "file-tail")]
let result = source::file_tail::run(&path, false, tx).await;
#[cfg(not(feature = "file-tail"))]
let result: Result<(), anyhow::Error> =
Err(anyhow::anyhow!("this build was compiled without the `file-tail` feature"));
if let Err(e) = result {
tracing::error!(error = %e, path = %path.display(), "extra file path exited with error");
}
})
}
fn source_kind_name(cfg: &config::SourceConfig) -> String {
@@ -418,6 +497,65 @@ async fn send_heartbeat(client: &mut LogIngestClient<Channel>, host: &str, servi
}
}
/// Same "no new proto, no new ingest code, no new ClickHouse schema"
/// shape as `send_heartbeat` above -- a metrics sample is just another
/// tagged `LogRecord`, distinguished by the `sentry.metrics` attribute.
/// Unlike heartbeat, the numeric fields themselves are real query-language
/// attributes too (`cpu_percent`, `mem_used_bytes`, etc.) rather than
/// being folded into `message` -- confirmed before building this that
/// arbitrary attribute names are transparently queryable/comparable as
/// numbers (`api/querylang/executor/sql.go`'s `topLevelFields` fallback
/// to `attributes['field']` with automatic numeric casting), so there's
/// no need to encode them as a JSON blob in `message` and parse it
/// client-side instead.
async fn send_metrics(client: &mut LogIngestClient<Channel>, host: &str, service: &str) {
let m = match metrics::collect("/").await {
Ok(m) => m,
Err(e) => {
tracing::warn!(error = %e, host, "collecting host metrics failed");
return;
}
};
let record = LogRecord {
timestamp_unix_nano: now_unix_nanos(),
host: host.to_string(),
service: service.to_string(),
severity: Severity::Info as i32,
message: "host metrics".to_string(),
attributes: std::collections::HashMap::from([
("sentry.metrics".to_string(), "true".to_string()),
("cpu_percent".to_string(), format!("{:.2}", m.cpu_percent)),
("mem_used_bytes".to_string(), m.mem_used_bytes.to_string()),
("mem_total_bytes".to_string(), m.mem_total_bytes.to_string()),
("disk_used_bytes".to_string(), m.disk_used_bytes.to_string()),
("disk_total_bytes".to_string(), m.disk_total_bytes.to_string()),
// Static-or-slow-changing context, not utilization numbers --
// sent on the same record so a viewer never has to correlate
// two different samples to make sense of the numbers above
// (see metrics::Metrics's doc comment).
("cpu_cores".to_string(), m.cpu_cores.to_string()),
("os_name".to_string(), m.os_name.clone()),
("kernel_version".to_string(), m.kernel_version.clone()),
("arch".to_string(), m.arch.to_string()),
("uptime_seconds".to_string(), m.uptime_seconds.to_string()),
// Comma-joined -- LogRecord attributes are string-valued,
// and a host can have more than one address per family
// (multi-NIC, or a v6 privacy/temporary address alongside
// the stable one). Empty string, not an omitted key, when
// a host genuinely has none of a given family -- matches
// every other soft-failed context field here (see
// metrics::collect's unwrap_or_default for this one).
("ipv4_addresses".to_string(), m.ipv4_addresses.join(",")),
("ipv6_addresses".to_string(), m.ipv6_addresses.join(",")),
]),
record_id: String::new(),
};
match grpc::send_batch(client, format!("metrics-{}", batch_id()), vec![record]).await {
Ok(_) => tracing::debug!(host, "metrics sent"),
Err(e) => tracing::warn!(error = %e, host, "metrics send failed"),
}
}
fn now_unix_nanos() -> i64 {
std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
+394
View File
@@ -0,0 +1,394 @@
use anyhow::{Context, Result};
use std::fs;
use std::process::Command;
use std::time::Duration;
pub struct Metrics {
pub cpu_percent: f64,
pub mem_used_bytes: u64,
pub mem_total_bytes: u64,
pub disk_used_bytes: u64,
pub disk_total_bytes: u64,
/// The rest of these are static-or-slow-changing context, not
/// utilization numbers -- sent alongside the utilization fields on
/// the same record (rather than as a separate one-off record) so a
/// viewer never has to correlate two different samples to answer
/// "is 21% CPU busy or idle for this box" (needs core count) or
/// "is this usage normal" (needs how long it's been running).
pub cpu_cores: u32,
pub os_name: String,
pub kernel_version: String,
pub arch: &'static str,
pub uptime_seconds: u64,
/// Non-loopback, non-link-local addresses only -- a host's `fe80::/10`
/// and `127.0.0.1`/`::1` are never what a viewer means by "this
/// host's IP", and would just add noise. Sorted and deduplicated,
/// but otherwise unfiltered: a multi-NIC host reports every address
/// it has, not just one "primary" guess (there's no reliable way to
/// pick a single "the" address from userspace without also knowing
/// which interface actually carries this host's traffic).
pub ipv4_addresses: Vec<String>,
pub ipv6_addresses: Vec<String>,
}
/// Collects a point-in-time snapshot of host resource usage plus the
/// system context needed to make it legible. Linux-only for now (a
/// disclosed gap, not a silent assumption -- see /agent/README.md):
/// every host this has been deployed to so far is Linux, and a Windows
/// implementation (perf counters/WMI) is real future work, not
/// attempted here.
pub async fn collect(disk_path: &str) -> Result<Metrics> {
let cpu_percent = cpu_percent().await.context("reading CPU usage")?;
let (mem_used_bytes, mem_total_bytes) = memory().context("reading memory usage")?;
let (disk_used_bytes, disk_total_bytes) = disk(disk_path).context("reading disk usage")?;
// Soft-fail on all four: none of these should ever cost a whole
// sample (losing real cpu_percent/mem/disk numbers) just because
// e.g. /etc/os-release is missing on some minimal distro --
// consistent with this codebase's existing "an optional feature's
// failure must never take down the thing it's supplementing"
// posture (see send_heartbeat/CheckIn's own graceful degradation).
let cpu_cores = cpu_cores().unwrap_or(0);
let os_name = os_name().unwrap_or_else(|_| "unknown".to_string());
let kernel_version = kernel_version().unwrap_or_else(|_| "unknown".to_string());
let uptime_seconds = uptime_seconds().unwrap_or(0);
let (ipv4_addresses, ipv6_addresses) = ip_addresses().unwrap_or_default();
Ok(Metrics {
cpu_percent,
mem_used_bytes,
mem_total_bytes,
disk_used_bytes,
disk_total_bytes,
cpu_cores,
os_name,
kernel_version,
arch: std::env::consts::ARCH,
uptime_seconds,
ipv4_addresses,
ipv6_addresses,
})
}
/// Two `/proc/stat` samples ~200ms apart, delta-based -- the standard
/// technique every `top`-like tool uses, since a single snapshot of
/// cumulative jiffies-since-boot can't express a percentage on its own.
/// Self-contained (no state threaded through main.rs's `select!` loop)
/// at the cost of blocking this one `collect()` call for ~200ms once
/// per `metrics.interval` tick -- an acceptable trade against the
/// complexity of holding a previous-sample struct across ticks in an
/// already-busy loop, for a feature that only runs once a minute by
/// default.
async fn cpu_percent() -> Result<f64> {
let (total1, idle1) = read_proc_stat()?;
tokio::time::sleep(Duration::from_millis(200)).await;
let (total2, idle2) = read_proc_stat()?;
let total_delta = total2.saturating_sub(total1);
let idle_delta = idle2.saturating_sub(idle1);
if total_delta == 0 {
return Ok(0.0);
}
Ok((1.0 - (idle_delta as f64 / total_delta as f64)) * 100.0)
}
fn read_proc_stat() -> Result<(u64, u64)> {
let contents = fs::read_to_string("/proc/stat").context("reading /proc/stat")?;
parse_proc_stat(&contents)
}
/// Parses `/proc/stat`'s leading "cpu " line: user nice system idle
/// iowait irq softirq steal guest guest_nice, all in USER_HZ jiffies
/// since boot. Returns (total, idle) -- idle here is idle+iowait,
/// matching what every standard CPU%-from-/proc/stat implementation
/// treats as "not busy" (iowait is a CPU waiting on I/O, not doing
/// work, even though the kernel's own `idle` field alone doesn't
/// include it).
fn parse_proc_stat(contents: &str) -> Result<(u64, u64)> {
let line = contents
.lines()
.find(|l| l.starts_with("cpu "))
.context("/proc/stat has no leading \"cpu \" line")?;
let fields: Vec<u64> = line.split_whitespace().skip(1).filter_map(|f| f.parse().ok()).collect();
if fields.len() < 4 {
anyhow::bail!("unexpected /proc/stat format: {line:?}");
}
let idle = fields[3] + fields.get(4).copied().unwrap_or(0);
let total: u64 = fields.iter().sum();
Ok((total, idle))
}
fn memory() -> Result<(u64, u64)> {
let contents = fs::read_to_string("/proc/meminfo").context("reading /proc/meminfo")?;
parse_meminfo(&contents)
}
/// Parses `/proc/meminfo`'s MemTotal/MemAvailable (kB). MemAvailable
/// (not MemFree) is the kernel's own "how much could a new process
/// actually get" estimate, accounting for reclaimable caches/buffers --
/// what a human means by "memory used" far better than MemFree alone
/// (a system with most of RAM in disk cache but MemFree near zero is
/// not actually under memory pressure).
fn parse_meminfo(contents: &str) -> Result<(u64, u64)> {
let mut total_kb = None;
let mut available_kb = None;
for line in contents.lines() {
if let Some(v) = line.strip_prefix("MemTotal:") {
total_kb = parse_meminfo_kb(v);
} else if let Some(v) = line.strip_prefix("MemAvailable:") {
available_kb = parse_meminfo_kb(v);
}
}
let total_kb = total_kb.context("MemTotal not found in /proc/meminfo")?;
let available_kb = available_kb.context("MemAvailable not found in /proc/meminfo")?;
let used_kb = total_kb.saturating_sub(available_kb);
Ok((used_kb * 1024, total_kb * 1024))
}
fn parse_meminfo_kb(s: &str) -> Option<u64> {
s.trim().trim_end_matches("kB").trim().parse().ok()
}
fn disk(path: &str) -> Result<(u64, u64)> {
let output = Command::new("df").arg("-B1").arg(path).output().context("running df")?;
if !output.status.success() {
anyhow::bail!("df exited with status {}: {}", output.status, String::from_utf8_lossy(&output.stderr));
}
parse_df_output(&String::from_utf8_lossy(&output.stdout))
}
/// Shells out to `df` rather than linking a statvfs binding -- same
/// "shell out to a boring, ubiquitous tool rather than add a dependency
/// or FFI binding" precedent `source/journald.rs` already sets for
/// `journalctl` (see /agent/README.md's "Why journalctl, not
/// libsystemd"). `-B1` requests byte-granularity output instead of the
/// default 1K-block units, so no unit conversion is needed here. Total
/// is `used + available`, not the raw block count `df` also reports --
/// some filesystems (ext4's default ~5% root reservation) hold back
/// blocks a normal process can never use, which would make a "percent
/// full" computed against the raw total look artificially low; `used +
/// available` matches what `df`'s own `Use%` column is computed
/// against.
fn parse_df_output(stdout: &str) -> Result<(u64, u64)> {
let data_line = stdout.lines().nth(1).context("df produced no data line")?;
let fields: Vec<&str> = data_line.split_whitespace().collect();
// Filesystem, 1B-blocks, Used, Available, Use%, Mounted on
if fields.len() < 4 {
anyhow::bail!("unexpected df output: {data_line:?}");
}
let used: u64 = fields[2].parse().context("parsing df's Used column")?;
let available: u64 = fields[3].parse().context("parsing df's Available column")?;
Ok((used, used + available))
}
fn cpu_cores() -> Result<u32> {
let contents = fs::read_to_string("/proc/cpuinfo").context("reading /proc/cpuinfo")?;
parse_cpuinfo_core_count(&contents)
}
/// Counts `processor\t: N` lines in `/proc/cpuinfo` -- one per logical
/// CPU (a hyperthreaded core counts as two, same as what `nproc`/every
/// scheduler-facing tool means by "CPU count"), which is what
/// `cpu_percent`'s 0-100 scale is an average across.
fn parse_cpuinfo_core_count(contents: &str) -> Result<u32> {
let n = contents.lines().filter(|l| l.starts_with("processor")).count() as u32;
if n == 0 {
anyhow::bail!("no \"processor\" lines found in /proc/cpuinfo");
}
Ok(n)
}
fn os_name() -> Result<String> {
let contents = fs::read_to_string("/etc/os-release").context("reading /etc/os-release")?;
parse_os_release_pretty_name(&contents)
}
/// Parses `/etc/os-release`'s `PRETTY_NAME="..."` line (e.g. "Debian
/// GNU/Linux 13 (trixie)") -- the one field every distro's os-release
/// is guaranteed to carry for exactly this "show a human a readable OS
/// name" purpose (see os-release(5)).
fn parse_os_release_pretty_name(contents: &str) -> Result<String> {
contents
.lines()
.find_map(|l| l.strip_prefix("PRETTY_NAME="))
.map(|v| v.trim().trim_matches('"').to_string())
.context("PRETTY_NAME not found in /etc/os-release")
}
/// `/proc/sys/kernel/osrelease` is just the bare version string (e.g.
/// "6.12.90+deb13.1-amd64") with no parsing needed -- simpler and more
/// robust than picking the version back out of `/proc/version`'s
/// free-form `uname -a`-style sentence.
fn kernel_version() -> Result<String> {
Ok(fs::read_to_string("/proc/sys/kernel/osrelease")
.context("reading /proc/sys/kernel/osrelease")?
.trim()
.to_string())
}
fn uptime_seconds() -> Result<u64> {
let contents = fs::read_to_string("/proc/uptime").context("reading /proc/uptime")?;
parse_uptime(&contents)
}
/// `/proc/uptime`'s first field is seconds since boot (as a float, to
/// centisecond precision) -- the second field (total idle time summed
/// across all cores) isn't relevant here.
fn parse_uptime(contents: &str) -> Result<u64> {
let first = contents.split_whitespace().next().context("/proc/uptime is empty")?;
let seconds: f64 = first.parse().context("parsing /proc/uptime's first field")?;
Ok(seconds as u64)
}
fn ip_addresses() -> Result<(Vec<String>, Vec<String>)> {
let output = Command::new("ip").arg("-o").arg("addr").arg("show").output().context("running ip addr show")?;
if !output.status.success() {
anyhow::bail!("ip exited with status {}: {}", output.status, String::from_utf8_lossy(&output.stderr));
}
Ok(parse_ip_addr_output(&String::from_utf8_lossy(&output.stdout)))
}
/// Shells out to `ip -o addr show` -- same "boring, ubiquitous tool"
/// precedent `disk`'s `df` call and `source/journald.rs`'s `journalctl`
/// call already set, over an FFI binding to `getifaddrs(3)`. `-o`
/// (oneline) puts each address on its own line, e.g.:
/// 2: eth0 inet 172.239.44.244/24 brd ... scope global eth0\ ...
/// 2: eth0 inet6 fe80::1/64 scope link \ ...
/// Skips the loopback interface by name (`lo`) and any address whose
/// line mentions `scope link` (IPv6 link-local, `fe80::/10`) or
/// `scope host` (loopback addresses `ip` sometimes reports even on a
/// non-`lo` line) -- neither is what a viewer means by "this host's
/// IP". Field 1 is the interface name, field 2 is the address family
/// (`inet`/`inet6`), field 3 is `address/prefix-length`.
fn parse_ip_addr_output(stdout: &str) -> (Vec<String>, Vec<String>) {
let mut v4 = Vec::new();
let mut v6 = Vec::new();
for line in stdout.lines() {
let fields: Vec<&str> = line.split_whitespace().collect();
if fields.len() < 4 {
continue;
}
let iface = fields[1];
if iface == "lo" || line.contains("scope link") || line.contains("scope host") {
continue;
}
let addr = fields[3].split('/').next().unwrap_or(fields[3]);
match fields[2] {
"inet" => v4.push(addr.to_string()),
"inet6" => v6.push(addr.to_string()),
_ => {}
}
}
v4.sort();
v4.dedup();
v6.sort();
v6.dedup();
(v4, v6)
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn parses_proc_stat() {
let contents = "cpu 100 0 50 800 20 0 0 0 0 0\ncpu0 100 0 50 800 20 0 0 0 0 0\n";
let (total, idle) = parse_proc_stat(contents).unwrap();
// total = 100+0+50+800+20 = 970; idle = 800 (idle) + 20 (iowait) = 820
assert_eq!(total, 970);
assert_eq!(idle, 820);
}
#[test]
fn rejects_proc_stat_with_no_cpu_line() {
assert!(parse_proc_stat("not cpu data\n").is_err());
}
#[test]
fn parses_meminfo() {
let contents = "MemTotal: 16384000 kB\nMemFree: 1000000 kB\nMemAvailable: 8192000 kB\n";
let (used, total) = parse_meminfo(contents).unwrap();
assert_eq!(total, 16384000 * 1024);
assert_eq!(used, (16384000 - 8192000) * 1024);
}
#[test]
fn rejects_meminfo_missing_fields() {
assert!(parse_meminfo("MemTotal: 16384000 kB\n").is_err());
}
#[test]
fn parses_df_output() {
let stdout = "Filesystem 1B-blocks Used Available Use% Mounted on\n/dev/sda1 80000000000 20000000000 60000000000 25% /\n";
let (used, total) = parse_df_output(stdout).unwrap();
assert_eq!(used, 20000000000);
assert_eq!(total, 20000000000 + 60000000000);
}
#[test]
fn rejects_df_output_with_no_data_line() {
assert!(parse_df_output("Filesystem 1B-blocks Used Available Use% Mounted on\n").is_err());
}
#[test]
fn counts_cpuinfo_processors() {
let contents = "processor\t: 0\nmodel name\t: x\n\nprocessor\t: 1\nmodel name\t: x\n";
assert_eq!(parse_cpuinfo_core_count(contents).unwrap(), 2);
}
#[test]
fn rejects_cpuinfo_with_no_processor_lines() {
assert!(parse_cpuinfo_core_count("model name: x\n").is_err());
}
#[test]
fn parses_os_release_pretty_name() {
let contents = "NAME=\"Debian GNU/Linux\"\nPRETTY_NAME=\"Debian GNU/Linux 13 (trixie)\"\nVERSION_ID=\"13\"\n";
assert_eq!(parse_os_release_pretty_name(contents).unwrap(), "Debian GNU/Linux 13 (trixie)");
}
#[test]
fn rejects_os_release_missing_pretty_name() {
assert!(parse_os_release_pretty_name("NAME=\"Debian\"\n").is_err());
}
#[test]
fn parses_uptime() {
assert_eq!(parse_uptime("12345.67 98765.43\n").unwrap(), 12345);
}
#[test]
fn rejects_empty_uptime() {
assert!(parse_uptime("").is_err());
}
#[test]
fn parses_ip_addr_output_excluding_loopback_and_link_local() {
let stdout = concat!(
"1: lo inet 127.0.0.1/8 scope host lo\\ valid_lft forever preferred_lft forever\n",
"1: lo inet6 ::1/128 scope host \\ valid_lft forever preferred_lft forever\n",
"2: eth0 inet 172.239.44.244/24 brd 172.239.44.255 scope global eth0\\ valid_lft forever preferred_lft forever\n",
"2: eth0 inet6 2600:3c06::1/64 scope global dynamic mngtmpaddr noprefixroute \\ valid_lft forever preferred_lft forever\n",
"2: eth0 inet6 fe80::abcd/64 scope link \\ valid_lft forever preferred_lft forever\n",
);
let (v4, v6) = parse_ip_addr_output(stdout);
assert_eq!(v4, vec!["172.239.44.244".to_string()]);
assert_eq!(v6, vec!["2600:3c06::1".to_string()]);
}
#[test]
fn parses_ip_addr_output_dedupes_and_sorts_multiple_interfaces() {
let stdout = concat!(
"2: eth0 inet 10.0.0.5/24 scope global eth0\\ valid_lft forever preferred_lft forever\n",
"3: eth1 inet 10.0.0.2/24 scope global eth1\\ valid_lft forever preferred_lft forever\n",
"3: eth1 inet 10.0.0.5/24 scope global secondary eth1\\ valid_lft forever preferred_lft forever\n",
);
let (v4, _v6) = parse_ip_addr_output(stdout);
assert_eq!(v4, vec!["10.0.0.2".to_string(), "10.0.0.5".to_string()]);
}
#[test]
fn parses_ip_addr_output_with_no_addresses() {
let (v4, v6) = parse_ip_addr_output("1: lo inet 127.0.0.1/8 scope host lo\\ valid_lft forever preferred_lft forever\n");
assert!(v4.is_empty());
assert!(v6.is_empty());
}
}