#38 cannot be made green from this side. No @sveltejs/kit release
accepts TypeScript 7: 2.70.3, the current latest, peers
`^5.3.3 || ^6.0.0`, and even the 3.0.0-next.27 prerelease peers
`^6.0.0`. svelte-check caps at 6 as well. `npm ci` stops on ERESOLVE
before a line is compiled, so there is nothing to fix here -- the block
is upstream.
Left open, Dependabot reopens it every week for as long as that stays
true. So the major is ignored for typescript in /web only, with the
condition for removing it written next to it. Minor and patch updates
still come through, and every other package is untouched.
#31 and #36 bump tonic-build alone to 0.14 and break the build outright:
0.14 moved prost codegen out to tonic-prost-build, so
`tonic_build::configure()` no longer exists. Both repositories also pin
`tonic = "0.12"` as the runtime beside it, and codegen from 0.14 against
a 0.12 runtime would be incoherent even if it compiled.
So the whole set moves together, in agent and in search:
tonic 0.12 -> 0.14
tonic-build 0.12 -> tonic-prost-build 0.14 (build-dependency)
prost 0.13 -> 0.14
tonic-prost new runtime dependency
Two consequences worth naming. The generated code now reaches for
`tonic_prost::ProstCodec`, so tonic-prost has to be a real dependency
rather than something the build script pulls in. And prost had to move
with it: tonic-prost 0.14 wants prost 0.14, so leaving ours at 0.13
left the generated types deriving a `Message` trait from a different
prost than the one the codec demanded.
tonic's `tls` feature is gone, replaced by one feature per crypto
provider. `tls-ring` is the like-for-like choice: it is the rustls-plus-
ring pairing `tls` used to mean, and the agent hands its own CA and
identity to ClientTlsConfig for mTLS, so it needs no root store.
The call sites did not change at all -- transport::{Certificate,
Channel, ClientTlsConfig, Identity} and include_proto! are all still
where they were.
Verified by building both, since no CI job compiles Rust: agent and
search check clean, 34 agent tests and 24 search tests pass.
gridstack 13 changed `GridStack.init` to return `GridStack | null` where
11 always handed one back. The dashboard held the result in a
`GridStack | undefined` and called `.on('change')` on it straight after,
so svelte-check stopped on two errors: null is not undefined, and the
value is possibly neither.
Coalesced to undefined so the declared type stays as it was, then
guarded before the listener is attached. `gridEl` is already checked at
the top of the function, so a null here should not occur -- but the type
allows it, and a dashboard that quietly stops persisting drags beats one
that throws inside an effect.
Nothing else in the repository touches gridstack: one import of
`GridStack` and its stylesheet, in this file.
Worth noting `vite build` passes either way. Only `npm run check` sees
this, and no CI job runs it -- the two majors would have gone in
looking clean.
Two Dependabot PRs are stuck behind the same number.
#35 raises the go directive to 1.26.0 in six modules, because
golang.org/x/crypto v0.56.0 requires it -- x/crypto tracks the two most
recent Go releases and 0.56 dropped 1.25. A module that says 1.26 cannot
be built by the 1.25 this repository pins in two places, so that PR
fails every Go job.
#29 raises actions/setup-go to v7, which sets GOTOOLCHAIN=local. With
that set, `go install golang.org/x/vuln/cmd/govulncheck@latest` cannot
quietly fetch a newer toolchain, and stops with
golang.org/x/[email protected] requires go >= 1.26.0 (running go 1.25.14)
Under setup-go v5 the same install succeeded by downloading 1.26 behind
our backs, which is its own reason to be on 1.26 deliberately instead.
So: security-scan's go-version and all eight Dockerfiles move together,
1.25 -> 1.26. Nothing else needs to. A newer toolchain builds an older
directive happily, so this stands on its own before #35 lands, and the
go.mod files stay where they are here.
Checked by building rather than by reading: the api and ingest images
both build on golang:1.26-alpine, and api, ingest and enterprise still
`go build ./...` clean against their existing 1.25 directives.
The account behind it was renamed from LINUXexpert-org to jcoffey-dev,
and GitHub does not redirect the old name: github.com/sponsors/
LINUXexpert-org answers 404 while the new one answers 200. So the
Sponsor button on this repository has been leading nowhere.
Worth fixing rather than leaving to redirect, because a released
username can be registered by anyone -- a stale link stops being a dead
end and starts being someone else's page.
GHSA-2v4p-qf9q-27wj is a panic in gRPC-Go's xDS routing interceptor: a
request arriving with neither `:authority` nor `Host` indexes an empty
slice, the per-RPC goroutine does not recover, and the process dies.
High, and nine alerts, because nine go.mod files pin the same version --
eight directly, terraform indirectly.
Nothing here was reachable. The interceptor is installed by
`xds.NewGRPCServer`, which this repo never calls: the one production
server is `grpc.NewServer(grpc.Creds(...))` in ingest/internal/grpcserver
and the only other is a plain one in a searchclient test. That is also
why security-scan has been green throughout -- govulncheck reports on
reachability and found nothing on 1.83.1, while Dependabot reports on
version ranges and found nine. Both were right.
Taken anyway: it is a patch release, and the next advisory in this
dependency may well land somewhere we do reach.
`go mod tidy` carried the indirect requirements grpc 1.83.2 asks for --
x/net, x/text, x/sys and friends. No CI job builds or tests Go here, so
all nine modules were built locally and api, ingest and enterprise
tested with -count=1, since a cached pass would not have exercised the
new version.
The dependabot.yml is the other half. There was no config, so nothing
opened a PR against any of this. Go majors stay out of the group, being
import path changes rather than bumps.
Dependabot #10: lru's IterMut violates Stacked Borrows, fixed in 0.16.3.
lru was transitive through tantivy 0.22.1, which pins lru ^0.12.0, so
there was no in-range fix -- cargo update -p lru locks nothing. The
advisory was also not reachable: tantivy calls only get, put, len,
peek_lru and new on its LruCache, never iter_mut. Upgrading rather than
dismissing because it is early enough that carrying four versions of
drift costs more than paying it now, and the alert then closes on its
own evidence rather than on an argument.
lru resolves to 0.16.4, past the patch line.
One API change across the four releases. TopDocs no longer implements
Collector on its own -- an ordering has to be chosen rather than
defaulted into. order_by_score() is exactly what bare TopDocs did in
0.22, so result order is preserved rather than quietly changed, which
matters for a search endpoint whose contract is "most relevant first".
Index compatibility checked rather than assumed, since a format change
would have meant a reindex for every existing deployment. Against a live
index of 1757 documents written by 0.22: the service opened it without
error, a document indexed hours earlier by 0.22 is still findable, new
documents written by 0.26 are findable, and a phrase query spans both.
No migration needed.
The compliance inventory is regenerated for the new graph: 17 crates
added, 5 gone, 20 bumped, and three duplicate-version entries collapsed
where the graph no longer needs two. Nothing newly flagged -- every
addition is permissive -- and cargo-deny check licenses, which is the
gate CI actually runs, passes. Rows for crates whose name and version
are unchanged are left byte-for-byte alone, so the diff shows the real
change rather than 200 rows of SPDX term reordering.
Signed-off-by: John Coffey <[email protected]>
The last unanswered action, and the only one whose output is not the
input with edits -- it emits a record that never existed, which is why
it was deferred twice.
It emits the window's first record unchanged, tagged with
cairnobs.aggregated, cairnobs.count, and the observed window bounds.
That follows the convention the agent already uses for heartbeat and
host-metrics records rather than inventing a second synthetic-record
mechanism, and keeping the first record intact means a reader sees a
real example of what was collapsed instead of an invented summary.
It tags even when the count is one. Emitting a bare record there would
be tidier and would make cairnobs.count present only sometimes, so
summing it silently breaks on quiet windows. window_last is the last
record that actually contributed, never window_start + window_ms,
because a window flushed early must not claim an end that never
happened.
Specifying it surfaced a problem the other nine actions do not have.
Windows are measured on record time, so a window can only be closed by a
later record arriving. suppress_duplicates never has anything pending;
aggregate_count holds state, so a matching stream that goes quiet leaves
its aggregate unemitted indefinitely -- data loss dressed as latency.
Emission therefore has a second trigger, end of stream, which the corpus
defines as an implicit flush after the last input and which production
gets from the batch flush. The cost is stated rather than hidden:
window_ms becomes a maximum, not a guarantee, and one burst can produce
more than one aggregate.
And it has a consequence nobody should meet in production first: stats
count undercounts aggregated data silently, so every panel and alert
counting rows changes meaning the moment a rule aggregates the data
behind it. Nothing here fixes that. The correct idiom is summing
cairnobs.count; teaching the query layer to do it automatically is a
Phase 2 change to the IR, recorded as the open question this decision
leaves in its place rather than quietly inherited.
Six cases added, corpus at 45. The validator's unspecified-action guard
stays in place with an empty set, still rejecting anything added to it.
Signed-off-by: John Coffey <[email protected]>
It was a defensible cut while a canary might have caught a bad rollout
partway through. With no canary it is the only thing that recovers a
host without an operator noticing, and "total evaluation should make it
unreachable" is what every crash-loop was before it happened.
Specified rather than named. A two-field file is written and fsynced
before rules are applied: the version being attempted, and whether it is
trying, good or quarantined. The rule set itself is never written, which
is what keeps the crash-loop-not-strand property -- an agent that loses
the file still boots clean and re-syncs. A version found still "trying"
at boot is quarantined, the agent starts with no rules, and it reports
the quarantined version so the failure is visible rather than merely
survived. A different version clears the quarantine, because pushing new
rules is the correction.
Three failure modes decided instead of discovered. An agent that cannot
write the file logs once and runs with total evaluation alone: degrading
to "no rules" would punish every read-only deployment for a failure that
has not happened, and the backstop is best-effort by construction rather
than by accident. An agent killed for an unrelated reason quarantines a
blameless rule set, which is a deliberate false positive -- the
alternative is claiming to distinguish "died because of the rules" from
"died while they happened to be loaded", which it cannot do honestly.
And a rule set fatal on only some hosts quarantines per host, which is
the closest thing to a canary this design has: the first host to hit it
reports while the rest carry on.
Also states that the conformance corpus cannot test any of this, so
nobody tries. The corpus pins rule semantics -- records in, records out.
Process death and file state across restarts are not expressible that
way and need agent-side tests driving a real process through crash and
restart.
Signed-off-by: John Coffey <[email protected]>
regex-lite on the agent, full regex at ingest, conformance corpus
limited to the syntax both accept. The agent had no regex dependency at
all and the full crate is a megabyte-plus against a binary whose pitch
is that it is small and static. The rejected alternative, regex
ingest-side only, would have given up redacting PII before it leaves the
host -- the one capability that most needs to be on the agent, and most
of the reason on-host processing exists at all.
Checked rather than assumed: every pattern the corpus uses compiles and
behaves under regex-lite, including named captures and replace_all
replacing every occurrence, which is what mask needs and what a
redaction stopping at the first hit would get wrong. Both engines also
resolve alternation leftmost-first, confirmed by running the same
pattern through regex-lite and Go's regexp and getting the same output.
A case now pins that, since it is the kind of semantic two independent
implementations can differ on silently.
No canary gate. An edit reaches every matching agent at once, and this
design had suggested that might have to block the feature. It does not,
but the risk is accepted rather than waved away: total evaluation is the
real mitigation, a fatal rule set crash-loops rather than strands
because overrides are never persisted, apply-then-verify makes that
self-healing, and applied_override_version already shows the blast
radius. What is genuinely given up is the ability to stop a bad rollout
partway through -- everything else shortens the outage without
preventing the rule reaching every host first.
That raises the stakes on apply-then-verify, which is still open.
Without a canary, total evaluation stops being a nice property of a
well-designed DSL and becomes the only thing between a bad rule and
every host at once, so cutting the backstop from v1 is a harder call
than it looked when it was written down.
Signed-off-by: John Coffey <[email protected]>
The design argues the conformance suite is the specification and should
be built before either implementation, since two hand-written
implementations of one language diverge unless something shared pins
them. This is that suite: 38 cases in /processing, a language-neutral
top-level directory for the same reason /proto is one -- the Rust agent
and the Go ingest tier both consume the definition and neither owns it.
Nothing executes the cases, because neither implementation exists. A
stdlib-only validator checks the corpus stays well-formed and runs in
CI: known actions, addressable fields, compilable patterns, names
matching filenames, and no case depending on record_id, which is
withheld so the question of whether an ingest-side rule can see one
stays open. The validator was checked against seven deliberately broken
cases before being trusted, since "38/38 valid" means nothing from a
validator that cannot fail.
Writing the cases first has already paid for itself twice.
It forced two determinism decisions the prose had left vague, both of
which a conformance suite cannot avoid answering. Sampling is
counter-based rather than random: random is statistically nicer and
impossible to assert on. Windows are measured on record timestamps
rather than wall-clock, which makes replay deterministic and, not
incidentally, makes backfill behave correctly where a wall-clock window
would not.
And it made the missing aggregate_count answer concrete. The design
does not say what that action emits, or what a query not expecting a
synthetic record sees. Rather than invent one by writing cases, the
validator rejects any case using it, so the design question has to be
answered before the behaviour can be frozen by accident.
The corpus includes the acceptance case from real measured data: the
two processes that account for roughly 60% of a real workstation's
journal volume, and the one kernel message worth keeping.
Signed-off-by: John Coffey <[email protected]>
Starts from the distribution channel and the safety invariant it
protects, and derives the language from them, rather than designing a
rule language and asking later how to ship it.
Four decisions proposed. A rule is a matcher plus ordered typed actions,
with no expressions and nothing resembling eval -- less expressive than
Cribl on purpose, and the only shape that can be pushed to ten thousand
hosts and audited by reading it. Total evaluation is the primary safety
guarantee, with apply-then-verify as a backstop. One spec with two
implementations means a language-neutral conformance suite is the
specification and should be built first, not last. Distribution reuses
DesiredOverride, following extra_file_paths as the precedent for a
repeated field.
It also corrects something #21 got wrong. That change said a fatal rule
would strand an agent the way a corrupted ingest endpoint does. Reading
apply_override's actual semantics, overrides live only in the running
process's memory and are never written to disk, so a restarted agent
boots clean and re-syncs -- a fatal rule set crash-loops rather than
strands, and the agent keeps checking in, so it stays correctable. That
is a much better failure mode, and it was acquired by accident: the
"don't persist" choice was made to avoid filesystem writes on read-only
images, not for safety. This design promotes it to a constraint, since
persisting overrides later would silently convert every crash-loop into
a strand. positioning.md is corrected to match rather than left
disagreeing.
Five open questions are left open rather than answered to look decisive,
the sharpest being that there is no staged rollout today: an edit
reaches every matching agent at once, which for executable rules is the
difference between breaking one host and breaking all of them.
The v1 acceptance test is real data, not a fixture: two processes on the
maintainer's own workstation account for 308 of 325 journal entries in
five minutes, and a suppress rule should remove about 60% of that host's
volume.
Signed-off-by: John Coffey <[email protected]>
Three of the roadmap's claims were contradicted by the repository
itself.
Fleet management was Phase 11, "Planned", and positioning.md said config
still flowed to the agent from the host rather than from the platform.
agent-management-design.md has recorded the opposite for some time:
punch list complete, verified live, with central authoring, versioning,
rollout on the next check-in, observation, and a restart command a real
agent picks up and acts on. There is no Phase 11 now. Its remainder is
either already named there -- stop/uninstall, per-host multi-row
alerting, a rule-per-host generator -- or belongs to Phase 8, since
distributing rules is the one genuinely new thing the mechanism has to
carry, and a rule language nobody can push to a fleet is not worth
having.
That inverts the old ordering argument, which put fleet last on the
grounds that it manages configuration the earlier phases define. Sound
reasoning; the world went the other way and built the mechanism first.
Recorded rather than quietly dropped, because the instinct behind it is
a good one that happened not to apply.
Retention was listed as a question Phase 10 would finally have to
answer. Half of it is answered: api/logretention serves operator-driven
preview and delete with an owner-only per-agent floor. What is missing
is an automatic TTL, so Phase 10 owns tiering and automatic TTL rather
than retention from nothing.
The rule-language recommendation is now settled rather than proposed,
and not on its own authority: DesiredOverride is already a closed typed
shape that cannot carry arbitrary code, so no channel exists that would
deliver JavaScript to an agent even if the language argument had gone
the other way.
That surfaced a requirement nothing had written down. Agent management
rests on an invariant it states outright -- every editable field
degrades behaviour without cutting off the agent's ability to receive
the next correction. Processing rules break it: a rule that panics or
loops strands the agent exactly the way a corrupted ingest endpoint
would, across every host it reached first. Phase 8 now owes either total
evaluation or apply-then-verify with rollback, chosen deliberately
rather than discovered mid-rollout.
Signed-off-by: John Coffey <[email protected]>
Cairn OBS is self-hosted, and the way to separate two environments is to
run two installations rather than two tenants inside one. Tenancy is the
wrong boundary for that, on three counts this repository demonstrates
rather than assumes: chwriter.WriteBatch is all-or-nothing across
tenants, so one tenant's failure stalls offset progress for every other;
CLICKHOUSE_DEFAULT_ACCESS_MANAGEMENT puts every tenant's data behind a
single superuser credential, as docker-compose.yml's own comment says;
and one binary with one set of migrations moves every tenant together,
which is the opposite of what separate environments are for. A whole
installation idles at about 1.3 GB, so the sharing buys nothing.
The project led with multi-tenant RBAC in the README banner and in
PROJECT-SPEC's goal statement. Both now say what it is instead:
self-hosted. "Open-core" goes with them -- it was already inaccurate,
since CONTRIBUTING states there is no feature gate and no paid tier, and
with enterprise/ off the roadmap there will not be one.
A second identity provider comes off the list of things standing between
this and production-ready. SSO belongs to enterprise/, and a self-hosted
deployment is not waiting on it. Terraform's tenant/RBAC resources move
from "disclosed future work" to not planned.
Nothing is scrubbed from the record. Phase 4 stays shipped, its runbook
stays, and its known gaps stay stated -- rewriting that history would
contradict the candour the Status section is built on. enterprise/ stays
in the tree, AGPLv3 and working, as the answer to a question this
project is not asking.
Signed-off-by: John Coffey <[email protected]>
Local login is implemented, wired through api, alerting and web, and
undiscoverable. No compose file turns it on, the Helm chart sets none
of its variables, and no markdown in the repository mentions
-seed-admin, LOCAL_AUTH_ENABLED or local login at all. The only way to
find it is to read cmd/api/main.go's authorizer switch.
Enabling it in docker-compose.yml is not the answer: a plain
`docker compose up` has no authentication, and every Phase 0-3 runbook
verifies the pipeline with bare curl against /query. Turning login on
by default would break the project's own documented verification.
So it's an opt-in overlay instead.
Four settings have to agree, and only one of them is obviously about
login. Each fails differently and none of the failures name the cause:
the route 404s, or the browser refuses the request before sending it,
or login returns 200 and every later request is anonymous because the
cookie was never stored, or the same symptom again from the opposite
end because the bundle never attaches it. That is what the new document
is mostly for.
The Helm chart still has no local-login support. Recorded in the
document as a gap rather than papered over.
Signed-off-by: John Coffey <[email protected]>
Phase 2 unified the two query languages behind one endpoint and renamed
the request field, and the runbooks were never updated. Following them
today does not work:
{"sql": ...} -> 400 query must not be empty
POST /api :8080/search -> 404, the route no longer exists
Both appear in the Phase 0 and Phase 1 runbooks and in the
windows-fixture README. That matters more than a normal doc typo,
because status.md cites the Phase 0 runbook as the record of how Phase
0 was verified -- so the documented verification procedure is one
nobody can re-run as written.
The Phase 1 step is rewritten rather than search-and-replaced: it
checked the SQL and full-text paths against two different endpoints,
and its exit criterion (the same record_id from both) now has to be
expressed against /query twice, once with SQL and once with a bare
word.
Phase 0's expected output for SELECT 1 also gained a warnings field
since it was written.
Every command here was run against a live stack before being written
down, including confirming both paths return the same record_id.
Signed-off-by: John Coffey <[email protected]>
jcoffey
jcoffey-dev