Add agent restart lifecycle command
Extends the existing CheckIn RPC with a one-shot AgentCommand (restart only -- stop/uninstall need real per-platform OS service-manager integration and stay deliberately out of scope), delivered at-most-once: cleared the instant it's handed to the agent in a response, since a restarting agent's process is gone before it could ever confirm receipt. On restart, the agent flushes whatever's buffered, aborts its source task, and exits cleanly, relying entirely on the host's own service manager to bring it back up. Issuing a command is gated at RoleAdmin (stricter than config editing's RoleEditor) and logged into the same audit_log table Phase 7's AI interactions use, via a new agent_command event type. A real bug was found and fixed during live verification: the first implementation tried to atomically read-and-clear pending_command in a single INSERT...ON CONFLICT statement using a sibling CTE referenced only from RETURNING, on the assumption that Postgres evaluates every part of a WITH query against one pre-statement snapshot. That's wrong specifically for FOR UPDATE, which always reads the latest row version including one written earlier in the same statement -- confirmed empirically (a restart command was always coming back empty even when genuinely pending, so the agent never received it). Fixed by splitting into two real, ordered statements inside one explicit transaction. See /docs/agent-management-design.md's "Lifecycle commands" section.
This commit is contained in:
@@ -20,7 +20,7 @@ use anyhow::{Context, Result};
|
||||
use batch::Batcher;
|
||||
use clap::Parser;
|
||||
use config::Config;
|
||||
use pb::agent::v1::{agent_control_client::AgentControlClient, CheckInRequest, DesiredOverride, ReportedConfig};
|
||||
use pb::agent::v1::{agent_control_client::AgentControlClient, AgentCommand, CheckInRequest, DesiredOverride, ReportedConfig};
|
||||
use pb::{log_ingest_client::LogIngestClient, LogRecord, Severity};
|
||||
use std::path::PathBuf;
|
||||
use std::time::Duration;
|
||||
@@ -171,6 +171,26 @@ pub async fn run_agent(config_path: Option<PathBuf>) -> Result<()> {
|
||||
tracing::info!(version = %applied_override_version, "applied remote config override");
|
||||
}
|
||||
}
|
||||
if AgentCommand::try_from(resp.pending_command) == Ok(AgentCommand::Restart) {
|
||||
tracing::info!("received remote restart command, shutting down gracefully");
|
||||
// flush_all(), not poll_timeout(): same reasoning as
|
||||
// the normal shutdown path below -- whatever's
|
||||
// buffered must go out regardless of whether
|
||||
// flush_interval has elapsed yet.
|
||||
if let Some(batch) = batcher.flush_all() {
|
||||
flush(&mut client, batch).await;
|
||||
}
|
||||
source_handle.abort();
|
||||
// A hard process exit, not a `break` out of this
|
||||
// loop: this agent's own restart policy is
|
||||
// entirely the host's service manager's
|
||||
// responsibility (systemd/Windows SCM), the same
|
||||
// contract any well-behaved service relies on --
|
||||
// see AgentCommand's doc comment for why STOP/
|
||||
// UNINSTALL need real per-platform work this
|
||||
// doesn't.
|
||||
std::process::exit(0);
|
||||
}
|
||||
}
|
||||
// A failed check-in is not fatal -- same graceful-
|
||||
// degradation posture as a failed heartbeat/batch
|
||||
|
||||
Reference in New Issue
Block a user