jcoffey-dev
bcb9a01cd6
The demo had 75k generic records across eight host-0N/service pairs, one dashboard, one alert rule, and -- because nothing ever called AgentControl.CheckIn -- a completely empty Agents page. /hack/demo-simulator replaces the generic data with a fictional but coherent fleet: 14 hosts running nginx, an API tier, workers, Postgres, Redis, mail, Linux journals and Windows event logs, whose messages and attributes look like what those services actually write. It backfills a week (~370k records, ~20s) and then keeps running. Running continuously is the point, not an implementation detail. Three things the demo has to show are only true if data keeps arriving: the Agents page marks a host stale once check-ins stop, alert rules evaluate over trailing windows and would freeze in one state against a static dataset, and any "last 15 minutes" view is empty on data that stopped growing overnight. It also emits metrics/heartbeats and answers CheckIn faithfully enough that the remote-config editor's pending -> applied transition works end to end. Seeded incidents give the data something to find: an api-02 outage with matching slow queries on db-01, 5xx at the edge and cascading job failures; an SSH probe burst; a spam wave; a disk filling up; and one decommissioned host left deliberately stale. /hack/demo-seed holds the rest of the deployment -- the nightly reset, eight dashboards (64 panels, every viz type but line), eleven alert rules across three notification targets, and the systemd unit. Rule thresholds are calibrated against what the simulator actually produces: the first pass had four rules whose thresholds the traffic could never reach and one that fired during normal operation. No line charts: dashboard panels reject the raw-SQL escape hatch, and the pipe language has no time-bucketing, so a real time axis isn't expressible today. Noted in demo-seed/README.md rather than papered over.
40 lines
1.3 KiB
Desktop File
40 lines
1.3 KiB
Desktop File
# The live half of the demo deployment: /hack/demo-simulator kept running
|
|
# so the demo's agents keep checking in, its hosts keep reporting metrics,
|
|
# and its logs keep arriving. -backfill 0 because history is seeded once
|
|
# per night by reset-demo.sh, which stops this unit for the duration and
|
|
# starts it again afterwards.
|
|
#
|
|
# Installed on the demo box at /etc/systemd/system/, not built into any
|
|
# image -- this is demo scaffolding, not part of the product.
|
|
[Unit]
|
|
Description=Cairn OBS demo data simulator
|
|
After=network-online.target docker.service
|
|
Wants=network-online.target
|
|
|
|
[Service]
|
|
Type=simple
|
|
User=john
|
|
WorkingDirectory=/home/john/cairnobs-demo
|
|
ExecStart=/home/john/cairnobs-demo/bin/demo-simulator \
|
|
-addr 127.0.0.1:4317 \
|
|
-ca /home/john/cairnobs-demo/hack/dev-certs/out/ca.pem \
|
|
-cert /home/john/cairnobs-demo/hack/dev-certs/out/client.pem \
|
|
-key /home/john/cairnobs-demo/hack/dev-certs/out/client-key.pem \
|
|
-backfill 0 -live
|
|
Restart=always
|
|
RestartSec=10
|
|
|
|
# Same unprivileged posture the real agents' units were moved to during
|
|
# the 2026-08-19 security remediation -- a data generator has no business
|
|
# with write access to anything on this box.
|
|
NoNewPrivileges=true
|
|
PrivateTmp=true
|
|
ProtectSystem=strict
|
|
ProtectHome=read-only
|
|
ProtectKernelTunables=true
|
|
ProtectControlGroups=true
|
|
RestrictSUIDSGID=true
|
|
|
|
[Install]
|
|
WantedBy=multi-user.target
|