8be2614364af8b9c78fadcd0883778705458828d
40
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 dependabot[bot]
|
37519c385a |
Bump the npm-minor-and-patch group in /web with 2 updates
Bumps the npm-minor-and-patch group in /web with 2 updates: [gridstack](https://github.com/gridstack/gridstack.js) and [vite](https://github.com/vitejs/vite/tree/HEAD/packages/vite). Updates `gridstack` from 13.2.0 to 13.3.0 - [Release notes](https://github.com/gridstack/gridstack.js/releases) - [Changelog](https://github.com/gridstack/gridstack.js/blob/master/doc/CHANGES.md) - [Commits](https://github.com/gridstack/gridstack.js/compare/v13.2.0...v13.3.0) Updates `vite` from 8.2.2 to 8.3.0 - [Release notes](https://github.com/vitejs/vite/releases) - [Changelog](https://github.com/vitejs/vite/blob/main/packages/vite/CHANGELOG.md) - [Commits](https://github.com/vitejs/vite/commits/[email protected]/packages/vite) --- updated-dependencies: - dependency-name: gridstack dependency-version: 13.3.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: npm-minor-and-patch - dependency-name: vite dependency-version: 8.3.0 dependency-type: direct:development update-type: version-update:semver-minor dependency-group: npm-minor-and-patch ... Signed-off-by: dependabot[bot] <[email protected]> |
||
 jcoffey-dev
|
efc8886bd8 |
Take gridstack to 13, and handle the grid it may not return
gridstack 13 changed `GridStack.init` to return `GridStack | null` where
11 always handed one back. The dashboard held the result in a
`GridStack | undefined` and called `.on('change')` on it straight after,
so svelte-check stopped on two errors: null is not undefined, and the
value is possibly neither.
Coalesced to undefined so the declared type stays as it was, then
guarded before the listener is attached. `gridEl` is already checked at
the top of the function, so a null here should not occur -- but the type
allows it, and a dashboard that quietly stops persisting drags beats one
that throws inside an effect.
Nothing else in the repository touches gridstack: one import of
`GridStack` and its stylesheet, in this file.
Worth noting `vite build` passes either way. Only `npm run check` sees
this, and no CI job runs it -- the two majors would have gone in
looking clean.
|
||
|
dependabot[bot]
|
be6614182f |
Bump gridstack from 11.5.1 to 13.2.0 in /web
Bumps [gridstack](https://github.com/gridstack/gridstack.js) from 11.5.1 to 13.2.0. - [Release notes](https://github.com/gridstack/gridstack.js/releases) - [Changelog](https://github.com/gridstack/gridstack.js/blob/master/doc/CHANGES.md) - [Commits](https://github.com/gridstack/gridstack.js/commits/v13.2.0) --- updated-dependencies: - dependency-name: gridstack dependency-version: 13.2.0 dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <[email protected]> |
||
|
dependabot[bot]
|
c0f0d51887 |
Bump the npm-minor-and-patch group in /web with 6 updates
Bumps the npm-minor-and-patch group in /web with 6 updates: | Package | From | To | | --- | --- | --- | | [@codemirror/commands](https://github.com/codemirror/commands) | `6.10.4` | `6.11.0` | | [@codemirror/state](https://github.com/codemirror/state) | `6.7.1` | `6.7.4` | | [@codemirror/view](https://github.com/codemirror/view) | `6.43.8` | `6.43.11` | | [@sveltejs/kit](https://github.com/sveltejs/kit/tree/HEAD/packages/kit) | `2.70.2` | `2.70.3` | | [svelte](https://github.com/sveltejs/svelte/tree/HEAD/packages/svelte) | `5.56.9` | `5.57.0` | | [vite](https://github.com/vitejs/vite/tree/HEAD/packages/vite) | `8.2.1` | `8.2.2` | Updates `@codemirror/commands` from 6.10.4 to 6.11.0 - [Changelog](https://github.com/codemirror/commands/blob/main/CHANGELOG.md) - [Commits](https://github.com/codemirror/commands/commits) Updates `@codemirror/state` from 6.7.1 to 6.7.4 - [Changelog](https://github.com/codemirror/state/blob/main/CHANGELOG.md) - [Commits](https://github.com/codemirror/state/commits) Updates `@codemirror/view` from 6.43.8 to 6.43.11 - [Changelog](https://github.com/codemirror/view/blob/main/CHANGELOG.md) - [Commits](https://github.com/codemirror/view/commits) Updates `@sveltejs/kit` from 2.70.2 to 2.70.3 - [Release notes](https://github.com/sveltejs/kit/releases) - [Changelog](https://github.com/sveltejs/kit/blob/version-3/packages/kit/CHANGELOG.md) - [Commits](https://github.com/sveltejs/kit/commits/@sveltejs/[email protected]/packages/kit) Updates `svelte` from 5.56.9 to 5.57.0 - [Release notes](https://github.com/sveltejs/svelte/releases) - [Changelog](https://github.com/sveltejs/svelte/blob/main/packages/svelte/CHANGELOG.md) - [Commits](https://github.com/sveltejs/svelte/commits/[email protected]/packages/svelte) Updates `vite` from 8.2.1 to 8.2.2 - [Release notes](https://github.com/vitejs/vite/releases) - [Changelog](https://github.com/vitejs/vite/blob/main/packages/vite/CHANGELOG.md) - [Commits](https://github.com/vitejs/vite/commits/v8.2.2/packages/vite) --- updated-dependencies: - dependency-name: "@codemirror/commands" dependency-version: 6.11.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: npm-minor-and-patch - dependency-name: "@codemirror/state" dependency-version: 6.7.4 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: npm-minor-and-patch - dependency-name: "@codemirror/view" dependency-version: 6.43.11 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: npm-minor-and-patch - dependency-name: "@sveltejs/kit" dependency-version: 2.70.3 dependency-type: direct:development update-type: version-update:semver-patch dependency-group: npm-minor-and-patch - dependency-name: svelte dependency-version: 5.57.0 dependency-type: direct:development update-type: version-update:semver-minor dependency-group: npm-minor-and-patch - dependency-name: vite dependency-version: 8.2.2 dependency-type: direct:development update-type: version-update:semver-patch dependency-group: npm-minor-and-patch ... Signed-off-by: dependabot[bot] <[email protected]> |
||
|
jcoffey-dev
|
dd5d5a77c3 |
Show the account controls the deployment actually has
The sidebar decided which auth mode was live from enterpriseAuthBase, so any deployment with VITE_ENTERPRISE_AUTH_BASE_URL set rendered the enterprise block -- and compose sets it unconditionally, so the tenant picker can exist. On a single-tenant stack with local login on, that meant the local block could never render: no username, no role, no Change password, no Log out, and in their place a "Sign in" link pointing at enterprise-auth's OIDC route, which is disabled unless OIDC_ISSUER_URL is configured. A dead link where the account controls should be. The build-time flag was never the right thing to ask. api registers /auth/* only when LOCAL_AUTH_ENABLED is set and ENTERPRISE_AUTH_URL is not, so the frontend cannot know the mode from its own build args -- the two can disagree, and here they did. getLocalSession already distinguishes 'disabled' (a 404 from /auth/session) from null (a 401, logged out); the sidebar collapsed both to null and threw the answer away. It now keeps that distinction and branches on it, so the mode comes from what the server actually serves. Logged out under local auth, the sidebar previously rendered no auth block at all -- no way back to the login page from the nav. It now offers Sign in, pointing at /login. Neither block renders until the probe lands, so nothing flashes the wrong mode on load. Signed-off-by: John Coffey <[email protected]> |
||
|
jcoffey-dev
|
200f801e2c |
Clear the Dependabot findings
Dependabot alerts were switched on for this repo today and reported 12 open findings. Ten are fixed here; the other two are addressed below. gRPC 1.83.0 -> 1.83.1, in all nine modules that require it. This is GHSA-vp52-pcj8-j9qc / CVE-2026-84304, heap memory exhaustion via HTTP/2 DATA frame fragmentation, affecting <= 1.83.0. It matters more than the version delta suggests: ingest/ is a gRPC listener deliberately exposed to the internet on :4317, so a remote OOM is reachable. mTLS narrows that to holders of a client certificate, which is why this was not an emergency, but the fix is one patch release away and there is no reason to carry it. golang.org/x/oauth2 0.21.0 -> 0.27.0 in deploy/operator, an indirect dependency (GHSA-6v2p-p543-phr9). enterprise/ was already past it at 0.36.0. npm cookie 0.6.0 -> 0.7.2, via an overrides entry rather than a dependency bump. @sveltejs/kit requires ^0.6.0 and still does at 2.70.3, the latest release, so there is no version of kit that resolves this on its own -- an override is the only route that does not involve waiting on upstream. Three incidental changes came out of `go mod tidy` and are not mine: genproto/googleapis/rpc moved forward as a transitive of the new grpc; pgx/v5 was reclassified from indirect to direct in enterprise/, which is simply correct, since audit.go and cmd/enterprise-auth import it; and the proto replace directive shuffled between require blocks at the same version. The twelfth finding, lru (GHSA-rhfx-m35p-ff5j), is not fixed and is not fixable here -- see the note in the pull request. It is CVSS 0, a Stacked Borrows soundness issue in IterMut, and reaching a patched version means tantivy 0.22 -> 0.26, which is a search engine migration rather than a dependency bump. Verified: all ten Go modules build, 40 test packages pass, the web app builds and svelte-check reports 0 errors across 288 files. |
||
|
jcoffey-dev
|
e8b6a8bc2e |
Answer 404 for unrouted paths, and keep the route lists honest
web/nginx.conf ended its try_files chain in an unconditional /200.html, so every path the site does not have -- /wp-login.php, /.env, a typo'd inbound link -- came back as the SPA shell with a success status. It now answers 404, which needs nginx to know which routes exist: most it infers from the build output, but dynamic routes and ones that never opted into prerendering have no file on disk and are listed by hand. Those hand-maintained lists drift, and the drift is invisible until it ships: vite dev and npm run preview route from the client manifest and never read nginx.conf, so a new dynamic route works everywhere a developer would look and 404s in production. hack/check-web-routes.sh compares the lists against web/src/routes, and a workflow runs it. Its own workflow rather than another job on license-compliance.yml, which already carries one unrelated check. Also turns absolute_redirect off. With nginx's default the trailing-slash canonicaliser reconstructs the origin from its own listen port, so a request for https://demo.cairnobs.org/settings/ was answered with Location: http://127.0.0.1:3000/settings -- the container's internal address, unreachable from the client, and downgraded to http on the way. Verified by curl against the built image; it was latent here before the canonicaliser existed too, through the directory redirect on /dev. |
||
|
jcoffey-dev
|
25d5d9ce2e |
Add the repository social preview card, with its derivation
GitHub's social card wants a 1280x640 raster and the brand package ships none. hero-grid.svg alone is the wrong source: it shows the mark with no project name, which is fine as a splash but wrong for a card that several platforms render without any accompanying repo title. Derived from two files already in the package rather than redrawn -- hero-grid.svg with its cairn group dropped for the background, and the package's own logo-horizontal-dark.svg composited on top so mark-to- wordmark spacing stays the designer's. The full recipe is recorded in web/src/lib/assets/README.md beside the existing note for the other derived asset, including why the crop is 96px off the top and 32px off the bottom rather than centred (the composition sits low and would crowd the bottom edge). Also noted there: the wide gaps in the wordmark are intended spacing, not a missing font -- the glyphs are outlined paths with no font-family in the file -- and re-running the recipe matches pixel for pixel but not byte for byte, since PNG writes a timestamp chunk. Not applied yet. GitHub exposes Settings -> General -> Social preview only on public repositories, and the section is absent while this one is private. |
||
|
jcoffey-dev
|
8e33f4cb0b |
Adopt the v2 "faceted glow" logo package, with the wordmark outlined
Swaps every brand asset the web UI imports for the v2 package: faceted stones with per-stone gradients and an ambient glow, replacing v1's flat four-colour paths. Raster favicons (16/32/48/180/512) are the package's own renders at the sizes already referenced, so the layout head is unchanged. Two things the swap needed beyond copying files in: - v2 ships no light-background stacked lockup, but the landing page picks one on theme. Derived it from the stacked-dark file the same way the package derives its own horizontal-light variant: same stone geometry and gradients, glow dropped, wordmark on light-surface ink/accent. - The stacked lockup is 360x320, no longer square; the landing page hard set an 11rem square, which stretched the wordmark. Height is auto now. The wordmark is outlined rather than left as live text. Upstream sets it in JetBrains Mono, but these load via <img src> and are inlined as data URIs, and an <img>-loaded SVG cannot reach the page's @font-face rules -- so neither JetBrains Mono nor our self-hosted Overpass Mono applied and the wordmark rendered in each visitor's default monospace, changing shape by platform. web/scripts/outline-wordmark.py converts every <text> to paths set in Overpass Mono Bold, the font the app already ships, so the lockup is pixel-stable and agrees with the UI's own type. Re-run it after refreshing the package, which will ship live <text> again. The four lockups now exceed Vite's 4KB inline threshold and ship as separate hashed immutable assets instead of data URIs duplicated across chunks. Unused package variants (icons, wordmarks, hero) are carried for completeness and imported nowhere, so they are not bundled. |
||
|
jcoffey-dev
|
e9ab528c08 |
Read query times in the display timezone, and stop capping result width
Two things that made reading logs harder than it needed to be. Query input: the API accepts an absolute time only if it is quoted AND carries an explicit offset, so someone reading logs in America/Denver who wanted "9am today" had to convert to UTC in their head and remember the quotes. Now a time typed without an offset is read as wall-clock time in that reader's display timezone and converted to the instant it names; anything with an explicit offset is taken at its word, and relative ranges never depended on a zone. This widens what's accepted rather than reinterpreting anything -- every naive form now handled is one the parser rejects outright today, so no query that works now can change meaning. The conversion happens before a query is sent *or saved*: a stored dashboard range becomes an explicit instant, because storing "2026-08-22 10:00" would mean 10am in whatever zone each viewer sat in, and one shared dashboard would show two people two different windows. It also fixes two bugs that predate the timezone work. injectTimeRange emitted absolute values unquoted, which the parser rejects -- so zooming a time-series chart into a range, and clicking a chart to drill down, both produced a syntax error on every panel. Both fed an ISO string straight into that unquoted path. Width: the query page's 64rem cap is gone, so the query bar and results table use the whole window -- a log table is the widest thing in this app and that cap was the horizontal scrolling. Prose keeps a readable measure, since full-width paragraphs are harder to read, not easier. Ambiguous local times -- the hour that repeats when clocks go back, the hour skipped when they go forward -- resolve to one instant. That is inherent to naming a moment by wall clock; an explicit offset sidesteps it. Documented at the conversion. |
||
|
jcoffey-dev
|
6ee918d15f |
Let each user pick the timezone timestamps are displayed in
Everything stays UTC: ingest still records Unix nanoseconds, ClickHouse
still stores UTC, every API response is still RFC3339 with a Z, and
queries are evaluated exactly as before. This changes only how those
instants are written on screen, so two people in two timezones looking
at one log line see the same instant written two ways -- never two
different lines, and never a different sort order.
Where the preference lives differs by deployment, and the three cases
are genuinely different products rather than one with fallbacks:
- Local login: server-side per named user (display_timezone on users,
PUT /auth/timezone), so it follows the person across browsers and
survives logout. Self-service at the RoleViewer floor, same as the
password change -- a viewer is the role most likely to be *only*
reading logs, so gating it higher would make it useless.
- Public demo: sessionStorage, so every new session starts at UTC. A
shared account's visitors have nothing to do with each other.
- Neither: localStorage, since there's no per-user record to write to.
api/cmd/api/main.go now imports time/tzdata. The image is
distroless/static with no /usr/share/zoneinfo, so LoadLocation would
otherwise reject every real zone name and the validation would refuse
every valid input.
Two details worth knowing when reading $lib/time.ts. Sub-second digits
are copied verbatim from the source string rather than round-tripped
through a JS Date, which is millisecond-precision and would silently
drop six digits of a ClickHouse nanosecond timestamp; expanding a result
row shows the localized value and the full-precision UTC original
together. And chart axes format their own labels, because ECharts'
type: 'time' axis renders in the browser's zone with no override --
which today puts a chart's clock out of step with the table beside it.
Timestamps are detected by value, not by column name: query output is
arbitrary, so a column called "timestamp" holding something else must
not be mangled, and `stats max(timestamp) as newest` must still be
formatted.
Verified against real zones including both sides of a DST boundary
(America/New_York at -05:00 in January, -04:00 in July), a half-hour
offset, and date rollover.
|
||
|
jcoffey-dev
|
04e83f64a9 |
Prefill the demo's login and say on its home page that it's a simulation
Two demo-only affordances, both off by default everywhere else. The login form starts with the demo's read-only account already in both fields, so a visitor doesn't need credentials handed to them out of band. It's a build-time opt-in: the web image is built with VITE_DEMO_USERNAME/VITE_DEMO_PASSWORD, and the page prefills only when it has both, so a deployment that sets neither -- every deployment except the demo -- gets the ordinary empty form, and a half-configured one can't leave a password next to an empty username box. This does bake a password into a static bundle, which is fine for exactly this case and nothing else: a Viewer-role account on a deployment whose database is wiped and reseeded nightly. api.ts says so next to the export, so nobody later points these at an account that can do something. The home page then explains what a visitor is actually looking at -- synthetic data from a simulated fleet, a nightly reset that discards anything they change, and the features that are deliberately limited (read-only account, alerts that notify a placeholder webhook, no time-series charts). Gated on the same signal as the prefill rather than a second flag that could drift out of sync with it. Also carries the landing page's light/dark logo swap, which touches the same file. |
||
|
jcoffey-dev
|
c920e0f2c4 |
Finish the Cairn OBS rename through services, docs, and assets
The rename commit before this one covered module paths and the obvious user-facing strings; this is the rest of it -- the places where "sentry" was a default value, a filename, or a picture rather than a word in a sentence. Defaults that changed: CLICKHOUSE_DATABASE (sentry -> cairnobs), POSTGRES_DATABASE (sentry_metadata -> cairnobs_metadata), and POSTGRES_USERNAME (sentry -> cairnobs), across api/alerting/ingest and the enterprise binaries, plus the compose files and migrate scripts that create those objects. These are *defaults*, so a deployment that sets them explicitly is unaffected -- but any deployment relying on the old defaults must have its environment updated before it picks this up, or it will come up pointing at a database that doesn't exist. Also: the light-mode logo variants (the dark ones existed alone, so the landing page and sidebar rendered a dark mark on a light background), regenerated favicons, and the docs/README/threat-model prose that still said Sentry. |
||
|
jcoffey-dev
|
13cf9a30cb |
Rebrand: Sentry -> Cairn OBS
Full rebrand across cosmetic branding, code identifiers, and infrastructure/data-plane naming, using the supplied Cairn OBS logo package. Cosmetic: favicon/logo swap (also closes a stale license-audit finding -- the old favicon was SvelteKit's unreplaced scaffold logo), new centered welcome landing page, larger/legible sidebar logo, page titles, CLAUDE.md/README/docs prose. Code identifiers: Go module path github.com/sentry/sentry -> github.com/cairnobs/cairnobs across all 13 modules and ~91 files (protoc regenerated); Rust crates sentry-agent/sentry-parser/sentry-search -> cairnobs-*; CLI sentryctl -> cairnobsctl; Terraform provider fully renamed (sentry_dashboard etc. -> cairnobs_dashboard, provider type, env vars); every session/auth cookie name; agent config paths and Windows service identity. Deliberately preserved: the gRPC wire protocol's protobuf packages (sentry.logs.v1, sentry.agent.v1) and their Go import directory (proto/sentry/...) -- renaming the wire-level package would break every currently-deployed agent binary (confirmed two real hosts, including mail.inbuxa.com, are actively streaming through this exact contract) until rebuilt and redeployed in lockstep with an ingest cutover. Only the Go module path wrapping the generated code changes. Infrastructure: every docker-compose container name (root and three component-level compose files); the Helm chart (directory, Chart.yaml, named-template helpers, all templates, values.yaml image repos); Kubernetes Operator (CRD group sentry.io -> cairnobs.io, both CRD YAML files, Go identifiers, RBAC markers); the coupled enterprise/tenantcrd package. Caught and fixed real path-coupling bugs along the way: the Helm chart's search/ingest volume mounts and the dev-only-credential detection constant vs. docker-compose.yml's literal values had to move together or a security warning would have silently stopped firing. Data plane: Postgres database sentry_metadata -> cairnobs_metadata and role sentry -> cairnobs; ClickHouse database sentry -> cairnobs; Kafka topic sentry.logs.raw -> cairnobs.logs.raw and its consumer groups. Source-level defaults, docker-compose.yml, and every migrate.sh/ provision script default updated together; already-applied migration files left untouched per this repo's immutable-migration convention. Verified at every layer: all 13 Go modules build/vet/test clean, both Rust workspaces (agent, search) build/clippy/test clean, npm run check/ build clean, docker compose config validates on all four compose files. Live-verified against a real docker stack multiple times through this work, including a final fresh-volume run confirming the actual renamed Postgres database/role, ClickHouse database, and Kafka topic all work end to end with a real login and query, zero console errors. |
||
|
jcoffey-dev
|
9e21ea17bb |
Fix log retention Settings section getting stuck on "Loading hosts…"
Root cause: handleHosts and partitionTargets both declared their result slices with `var`, so an empty result (no logs old enough yet, or every requested target blocked by a floor) marshaled to JSON `null` instead of `[]` on fields without `omitempty`. The frontend's `.length` access on that `null` threw mid-render, which is why this shipped with the spinner stuck forever instead of the empty state ever painting -- production is freshly deployed with nothing yet older than the default 30-day cutoff, so every user hit this on first load. Also replaces the static "Loading hosts…" text with the existing shimmer Skeleton component for real visual feedback, and adds `?? []` fallbacks in api.ts as a second line of defense. |
||
|
jcoffey-dev
|
653e4efa76 |
Enforce the full user-management RBAC matrix, add self-service password change
api/localauth now enforces every rule of the requested matrix, each
checked inside the handler beyond RegisterRoutes' floor:
- At least one owner must always exist -- handleDeleteUser and
handleSetRole both refuse an operation that would leave zero
owners (wouldRemoveLastOwner, backed by new store method
CountUsersWithRole), whether the caller is admin or owner.
- Owner can create/delete any role, including another owner (subject
to the above). Admin can only create/delete viewer or editor --
GET/POST /auth/users and DELETE .../{id} moved from RoleOwner to
RoleAdmin floor, with an inner check narrowing what an admin
caller specifically may target.
- Only a user can change their own password -- new POST
/auth/password (RoleViewer floor, i.e. every role) requires the
caller's current password (verified via new store method
GetPasswordHashByID) and is now the only path to changing your
own, including for an owner. The existing admin-reset endpoint
(POST /auth/users/{id}/reset-password, also moved to RoleAdmin
floor) now refuses id == the caller's own ID, and refuses an
owner target unless the caller is themselves an owner -- "admin
can change any password except an owner's; owner can change any
password, even another owner's."
- Role reassignment (PUT .../{id}/role) stays owner-only, unchanged
beyond the last-owner guard above.
New web/src/routes/account page (linked from NavSidebar next to "Log
out", visible to every local-auth role) is the self-service password
change UI. /users now mirrors the server's per-row restrictions
client-side (disabled role selects/delete/reset buttons with an
explanatory title, a restricted role list on the create form) so an
admin never sees an action that would just 403 -- the server remains
the actual authority.
Verified live against real Postgres and in the browser: the full
matrix via curl (owner creating a second owner, admin blocked from
creating/deleting/resetting admin or owner accounts, last-owner delete
and demote both blocked, admin resetting non-owner passwords,
self-target reset rejected, self-service change with wrong/right
current password), plus the actual /users page rendering correctly
restricted for an admin session and a full change-password round trip
through the real UI ending in a forced re-login with the new password.
|
||
|
jcoffey-dev
|
5ff2e5bb60 |
Scope log retention deletion and floors to (host, service), not host alone
logs rows carry a real per-record `service` (nginx, smtp, ufw, ...) -- already true of the schema (storage/migrations/0001) and wire protocol, not something this feature invents. Both the deletion picker and the retention floor now operate on (host, service) pairs instead of whole hosts, so an operator can delete just one noisy log type from an agent without touching everything else it ships, and can protect one service (e.g. keep smtp a year) longer than the rest of that host's default. api/agents.ConfigOverride gains ServiceLogRetentionDays (map[string]int), owner-only to change like LogRetentionDays -- a service listed there overrides the host's LogRetentionDays default for that service only. Agent config page gets a matching "Per-service log retention overrides" add/remove list next to the existing host-level field. api/logretention: Store's count/delete now take []HostService and build a ClickHouse tuple IN ((?,?),...) over (host, service); AgentRetentionStore. FloorsByHost returns each host's default plus its per-service map, with HostFloor.Effective(service) resolving which one applies. preview/delete moved from GET/DELETE-with-query-params to POST-with-JSON-body (a list of targets needs a real body, not a repeated compound query param), and partitionTargets checks the floor per target so one protected service never blocks deleting a different, unprotected one in the same request. Settings' Log retention section is a two-level picker now: each host row (with a "select all services" checkbox and its default floor badge) expands to its services, each with its own count and effective protected-days badge. Verified live against real ClickHouse/Postgres and in-browser: a host with a 7-day default plus a 365-day smtp override -- deleting nginx+ smtp+ufw together correctly removed nginx and ufw, left smtp's 10 records untouched, and confirmed via a follow-up owner delete that bypassing the floor works. Also verified the full click-through (add a service override on the agent page, see it reflected in Settings' picker, select/preview/cancel) and confirmed no regression from the prior host-only version's tests. |
||
|
jcoffey-dev
|
087c52a64f |
Scope log retention deletion to selected hosts, not the whole table
api/logretention no longer deletes wholesale by age alone: a new
GET /logs/retention/hosts lists every host with matching records (plus
any configured retention floor), and preview/delete now require an
explicit, non-empty host list -- there is no "omitted host means every
host" shortcut server-side. Store's count/delete statements are
host-scoped (host IN (...)); Handler.partitionHosts checks the floor
per host instead of one global max, so a floor on one host never
blocks acting on other hosts requested in the same call. A request
that ends up fully or partially blocked still returns 200 with
blocked_hosts explaining why, rather than rejecting the whole call.
Settings' Log retention section is a host picker now: checkboxes with
per-host counts and a "protected Nd" badge where a floor applies,
"select all/none", and a confirm panel that names exactly which hosts
will be affected and which were skipped and why.
Verified live against real ClickHouse/Postgres and in-browser: three
hosts seeded, one protected by a 90-day floor -- a scoped delete
correctly removed the two open hosts' records, left the protected
host's untouched, and the response/UI both named it as skipped. Also
fixed a real spacing bug in the result message caught during that
browser pass (an adjacent {expr}{#if} with no source whitespace
between them rendered with no space either).
|
||
|
jcoffey-dev
|
a20bb5d1c7 |
Add per-agent log retention floor, owner-only to set or override
api/agents.ConfigOverride gains LogRetentionDays: a per-agent setting edited on the same remote-config page as extra_file_paths, but unlike every other field there it's central-policy metadata api/logretention reads, never something the agent process itself sees. Any change to it -- setting, raising, lowering, or clearing -- requires RoleOwner, not just RoleAdmin: the whole point of the field is a floor an admin can't move, so an admin able to freely edit it would defeat that. api/logretention now checks the largest LogRetentionDays configured across any agent (AgentRetentionStore, new) before every preview/delete: a non-owner's request is rejected with a clear 403 if it would reach into that protected window. An owner always bypasses it, matching "make the log retention override any attempts to delete logs by anyone other than owner role." Verified live end-to-end: owner sets a 90-day floor on an agent, admin is blocked deleting anything newer than that (both preview and delete), allowed beyond it, and owner bypasses it entirely -- confirmed against real ClickHouse data, not just the fake-backed unit tests. Also caught and fixed a real pre-existing latent bug while verifying in-browser: a type="number" Input's bind:value becomes an actual JS number once a user types into it (only the initial value is a string), which broke a bare .trim() call on the new field. |
||
|
jcoffey-dev
|
787def06fd |
Add owner/admin-only log retention deletion to Settings
New api/logretention package: GET /logs/retention/preview and DELETE /logs/retention, both gated to RoleAdmin (Owner satisfies it too), issue purpose-built parameterized statements against ClickHouse's logs table (a count and a synchronous ALTER TABLE ... DELETE mutation) rather than routing through querylang/executor's SELECT-only SQLRunner. Settings gets a new "Log retention" section, visible only to an owner or admin, that previews how many records a chosen age cutoff would remove before showing an explicit confirm/cancel panel -- no delete happens without that second step. Scoped to core's single-tenant ClickHouse table; enterprise/'s per-tenant routing and Tantivy's lack of a bulk-delete primitive are disclosed gaps in api/logretention/store.go's doc comment, not silently assumed to already work. |
||
|
jcoffey-dev
|
864e68253a |
Give local users their own manager: custom passwords and role reassignment
Move user management out of Settings into its own /users page (nav-gated
to owners), let an owner type a specific password on reset instead of
always generating a random one, and add role reassignment via a new
PUT /auth/users/{id}/role endpoint. Role changes revoke the target's
existing sessions, same as a password reset, so a demoted user can't
keep acting under a stale, higher-privileged session.
|
||
|
jcoffey-dev
|
4b5dae5879 |
Add local login, agent extra log paths, IPv4/IPv6 metrics; remediate security audit findings
This is a large squashed commit covering two batches of prior uncommitted work plus a full security-audit remediation pass, kept together because go.mod/go.sum and several shared files (main.go, handler.go) were touched by both and splitting risked non-building intermediate commits. Features (built earlier, previously uncommitted): - Local username/password login for single-tenant deployments with no SSO configured (api/localauth, alerting/internal/sessioncheck, sentryctl users, web/src/routes/login, metadata migrations 0040/0041). - Remotely-editable additional log file paths for agents, on top of their existing primary source (api/agents, agent/sentry-agent extra-file-path diffing, web agent config UI). - IPv4/IPv6 addresses reported alongside other host system metrics. Security audit remediation (this pass, all live-verified in production): - Critical: block ClickHouse SSRF table functions (url/remote/file/s3/...) in the raw-SQL query escape hatch. - High: deny sensitive paths and require Admin to add agent extra_file_paths (Editor could previously point an agent at /etc/shadow or an SSH key); alerting webhook targets now validate against internal/metadata/loopback addresses, both at creation and send time; alerting's session middleware now enforces an Editor+ floor on mutating requests instead of "any authenticated session"; bumped goxmldsig to close a SAML signature-verification bypass (GO-2026-4753). - Medium: per-IP login rate limiting; security response headers (HSTS/CSP/nosniff/X-Frame-Options/Referrer-Policy/Permissions-Policy) on web/nginx.conf; a DevCredentialWarnings check in every Go service's config loader, logging loudly at startup if a deployment is still on docker-compose.yml's literal dev-only credentials; dependency bumps (golang.org/x/text, grpc, x/net, quick-xml, h2) across every affected Go module and both Rust crates, including a previously-uncovered x/net vulnerability in deploy/operator; a new security-scan.yml CI workflow running cargo-deny/govulncheck/npm-audit, mirroring the existing license-compliance.yml matrix shape. - Low: removed sentryctl's plaintext --password flag (shell history/`ps` exposure) in favor of stdin and a --password-stdin flag for reset-password's optional specific-password path; a dummy bcrypt comparison closes a login response-time username-enumeration side-channel. |
||
|
jcoffey-dev
|
93c160ec51 |
Add agent restart lifecycle command
Extends the existing CheckIn RPC with a one-shot AgentCommand (restart only -- stop/uninstall need real per-platform OS service-manager integration and stay deliberately out of scope), delivered at-most-once: cleared the instant it's handed to the agent in a response, since a restarting agent's process is gone before it could ever confirm receipt. On restart, the agent flushes whatever's buffered, aborts its source task, and exits cleanly, relying entirely on the host's own service manager to bring it back up. Issuing a command is gated at RoleAdmin (stricter than config editing's RoleEditor) and logged into the same audit_log table Phase 7's AI interactions use, via a new agent_command event type. A real bug was found and fixed during live verification: the first implementation tried to atomically read-and-clear pending_command in a single INSERT...ON CONFLICT statement using a sibling CTE referenced only from RETURNING, on the assumption that Postgres evaluates every part of a WITH query against one pre-statement snapshot. That's wrong specifically for FOR UPDATE, which always reads the latest row version including one written earlier in the same statement -- confirmed empirically (a restart command was always coming back empty even when genuinely pending, so the agent never received it). Fixed by splitting into two real, ordered statements inside one explicit transaction. See /docs/agent-management-design.md's "Lifecycle commands" section. |
||
|
jcoffey-dev
|
4f0da1ae5e |
Add agent inventory, management, and remote config
Extends the heartbeat mechanism with a second gRPC service on the same mTLS channel (AgentControl.CheckIn, agent-initiated on the existing heartbeat ticker -- still push-only, no inbound port on any agent) so an agent reports its running config and can pick up an operator-set override. A new web UI section (/agents) lists every agent that's checked in, shows its reported config, and lets an operator edit a narrow, deliberately-scoped subset remotely: batch/heartbeat tuning, and (journald sources only) the unit filter. TLS material and the ingest endpoint are never reportable or remotely editable, by proto shape rather than a validation rule -- a bad or malicious edit there could permanently strand an agent or redirect where its logs go, unlike every other editable field, which only degrades behavior. An override lives only in the agent's memory (agent.toml is never rewritten) and re-syncs on the agent's own schedule; changing the journald filter aborts and respawns the source task since there's no other way to change what's being tailed. Building the hot-reload path surfaced a real, independent, pre-existing bug: shutdown was using poll_timeout(), which only drains once flush_interval has elapsed, silently dropping anything buffered more recently on every graceful shutdown that landed between flushes -- fixed with a new unconditional Batcher::flush_all(), now used at both shutdown and hot-reload. Verified live end-to-end against a real stack: an edited heartbeat interval changed a running agent's actual send cadence within one check-in cycle (confirmed by the real timestamps landing in ClickHouse), and an edited journald filter triggered a real source restart, both reflected back in the next reported-config snapshot. See /docs/agent-management-design.md. |
||
|
jcoffey-dev
|
7d316f92db |
Phase 7: AI-assisted query authoring (autocomplete, explain, fix, optimize, NL translation)
Adds a self-hosted (Ollama, qwen2.5-coder) model provider abstraction with a pluggable opt-in cloud adapter, schema grounding, and a shared cost/safety guard every AI-suggested query is assessed against -- compiling to and executing through the same unchanged Phase 2 IR/ compiler and Phase 4 tenant scoping as a hand-written query, no parallel execution path. Track A (built into the query bar): inline ghost-text autocomplete, "Explain this query", "Fix this query" with a diff view, and a rule-based "Optimize" suggestion. Track B: natural-language-to-query translation, always a separate review step from execution, with `sentryctl query --nl` requiring explicit confirmation to run. Every accepted/dismissed translate-fix-optimize interaction is logged into the same append-only audit_log table Phase 4 built. Two real product bugs were found and fixed via live browser verification (a Svelte effect re-running on every keystroke that silently cancelled the ghost-text debounce; a ghost-text widget positioned at document offset 0 instead of the cursor), and a real costguard logic bug (unbounded-aggregation vs. raw-row) was caught by its own test suite. New integration tests wire a real Ollama client through the real HTTP handler against a mock server matching Ollama's wire contract (hack/mock-ollama), keeping model-quality verification out of CI as a disclosed, periodic human-run check instead. See /docs/phase-7-ai-design.md and /docs/phase-7-runbook.md. |
||
|
jcoffey-dev
|
661568085e |
Phase 6: license-compliance audit and enterprise/ relicensing to AGPLv3
Full dependency inventory across Rust/Go/npm plus Docker base images and vendored assets (776 rows, 502 unique deps), classified against AGPLv3 compatibility with real citations rather than assumptions. enterprise/ relicensed from its commercial-license stub to AGPLv3, matching core -- the one real flag (Redpanda's BSL 1.1) was evaluated against primary sources and accepted as-is rather than triggering a broker swap. CI enforcement wired up (.github/workflows/license- compliance.yml, this repo's first CI workflow), a root LICENSE file added, and every doc/comment referencing the old commercial-license boundary updated to describe it as architectural only. See /docs/compliance/ for the full report, inventory, and policy. |
||
|
jcoffey-dev
|
8ec370dcee |
Redesign alerting UI: severity-colored state and a delivery timeline
AlertStatePill reuses the log-severity color tiers instead of a second color vocabulary: ok -> quiet, pending -> warn, firing -> critical. DeliveryTimeline reframes the existing delivery_log data (no new backend fields) as a vertical timeline -- "why didn't I get paged" is a chronological question a flat table answered less directly. Rules list sorts firing-first. The list table's trailing actions column had a bare empty <th>, which axe-core flags (empty-table-header) -- fixed with visually-hidden text via app.css's shared .sr-only utility. |
||
|
jcoffey-dev
|
0e37ca6669 |
Redesign query/search: syntax highlighting, autocomplete, richer results
QueryEditor.svelte wraps CodeMirror 6, not a hand-rolled
textarea-plus-overlay highlighter -- autocomplete needs real
cursor-aware popup positioning a plain textarea can't give. language.ts
is a StreamLanguage tokenizer for the pipe grammar; its token() function
must return real @lezer/highlight tag names looked up by string
('controlKeyword', 'operatorKeyword', 'name.function' for tag+modifier
pairs) -- a custom Tag.define() looks plausible but silently highlights
nothing. completions.ts is context-aware: stage keywords after `|`,
stats functions after `stats`, field names elsewhere.
A two-way-binding race between the editor's updateListener and an
external-sync $effect could drop characters on rapid/bulk input --
fixed with a lastEmitted guard so the sync effect only reacts to
genuinely external value changes, not its own echoes.
ResultsTable gets sortable columns (a real <button> in the <th>, so
sorting is keyboard-operable for free), resizable columns
(pointer-drag, deliberately mouse-only -- the resize handle stays out
of the tab order, same as most apps treat column resize), and
expandable rows. The row-expand affordance was originally a bare `<tr
onclick>` with no keyboard equivalent at all; fixed with
tabindex/role="button"/aria-expanded and an Enter/Space handler.
AddToDashboardModal lets a query built on the Search page become a
saved panel without hand-copying the query string.
|
||
|
jcoffey-dev
|
45e0865a0c |
Rebuild dashboard panels on the new chart layer
Drag-and-drop grid stays on GridStack (already a Phase 3 dependency -- no new library needed). PanelEditor.svelte (a Modal) replaces the old inline add-panel form: a debounced live preview reuses PanelViz directly, so the preview is pixel-identical to what renders on save instead of drifting from a separate preview renderer. Dashboards list and detail pages get EmptyState/Skeleton for empty/loading states instead of a blank panel or a raw error string, and panel titles are now clickable buttons that open the editor. |
||
|
jcoffey-dev
|
5e8b3d8edd |
Add a real charting layer and a heatmap panel type
Five chart types on ECharts (modular imports, not the full bundle):
TimeSeriesChart (multi-series, legend toggle), BarChart (incl.
stacked), SingleStat (big number + sparkline + trend), Heatmap, TopN.
Shared interactions: tooltips, dataZoom feeding the global time-range
picker, click-to-drill-into-query (drilldown.ts strips a panel's query
to its pre-stats filter and appends the clicked series/x-value as a
new filter term -- no backend change needed).
pivot.ts reshapes the query language's existing {columns, rows} tabular
output into per-series chart data client-side -- `stats count by
service, timestamp` already returns "long" rows, so multi-series
support needed zero query-language changes. theme.ts reads real
computed CSS custom properties so charts render in the active theme's
actual colors, with an SSR_FALLBACK for adapter-static's prerender pass
where `document` doesn't exist.
heatmap is the one narrow, justified backend change: a new VizType
needed to feed a new visualization, not a new query capability. Three
places had to change together, not two -- api/dashboards/types.go's
validator, web/src/lib/api.ts's union (previous commit), and the
dashboard_panels table's viz_type CHECK constraint
(migrations/0035_add_heatmap_viz_type.sql), which mirrors the Go
validator and doesn't update itself.
/dev/charts (unlisted, dev-only) is a synthetic fixture/perf-test route:
confirmed 50ms first-two-frames render time on a production build
against a 30,006-row/6-series stress case, and a 211,975-byte gzipped
chart chunk -- both real measurements behind the ECharts-over-
Observable-Plot-or-D3 choice, not estimates.
|
||
|
jcoffey-dev
|
f1a68455e0 |
Build the ui/ component library and persistent app shell
Button, Input, Select, Badge, SeverityBadge, Table, Card, Modal, Tooltip, Tabs, Skeleton, EmptyState -- a shared library so pages stop hand-rolling markup per page (web/src/lib/components/ui, barrel export in index.ts). Modal and CommandPalette are built on native <dialog> for a real focus trap, Escape-to-close, and top-layer stacking instead of hand-rolling those. Tabs uses roving tabindex with arrow-key nav. NavSidebar replaces the old top-nav with a persistent sidebar (Search/Dashboards/Alerts/Data Sources/Settings), a live tenant indicator (api.ts's new getCurrentSession(), a client for the already-existing POST /internal/authorize -- zero new backend surface), theme/density quick-toggles, and a command-palette hint. Collapses to an off-canvas drawer under 860px. CommandPalette (Cmd/Ctrl+K) indexes the five static destinations plus live-fetched dashboards/alert rules. Data Sources is a new, honestly-scoped placeholder page (one data source per tenant today, no UI needed yet). Settings and Select-tenant are re-tokened onto the new component library. +layout.svelte's content wrapper is a plain <div>, not a second <main> -- every page already renders its own top-level <main>. |
||
|
jcoffey-dev
|
a6153c5f90 |
Build the Signal design system: tokens, fonts, theme, density
Self-hosted variable fonts (Overpass/Overpass Mono, OFL-licensed) -- no CDN dependency for the app to render correctly. Dark is the literal default in tokens.css (:root defines it directly; light is the override via both prefers-color-scheme and an explicit data-theme), not a retrofit. Severity tokens collapse OTel's seven severities to five visual tiers (severity.ts); their translucent -bg variants and light-mode warn's base color are already tuned for WCAG AA contrast against an opaque surface, not just the plain page background -- verified with real axe-core runs during the accessibility pass, see docs/design-system.md. theme.svelte.ts/density.svelte.ts persist to localStorage and expose getter/setter functions wrapping module-level $state (Svelte 5's shared-state-module pattern -- a directly exported $state doesn't preserve reactivity across modules). app.html's inline script applies both before first paint to avoid a flash of the wrong theme/density. |
||
|
jcoffey-dev
|
fb502f3d31 |
Add ECharts, CodeMirror, and axe-core for Phase 5
ECharts backs the new charting layer, the @codemirror packages back the query editor's syntax highlighting and autocomplete, and axe-core (dev-only) drives the automated accessibility sweep. Drops uplot, which the chart rebuild replaces entirely. |
||
|
jcoffey-dev
|
0452d1f921 |
Fix web/Dockerfile dropping two of its three VITE_* build args
Only VITE_API_BASE_URL had a matching ARG/ENV pair; docker-compose.yml's build args for VITE_ALERTING_API_BASE_URL and VITE_ENTERPRISE_AUTH_BASE_URL were silently dropped by Docker (an undeclared --build-arg is dropped, not an error). enterpriseAuthBase came out undefined in the built bundle, so the tenant-picker page threw "enterprise-auth is not configured" against a real running container even though docker-compose.yml looked correct. The other two vars masked this because web/src/lib/api.ts's apiBase/alertingBase both have hardcoded fallbacks that happen to match the intended values. Found while wiring a real Auth0 developer tenant into enterprise-auth to close §3a/§12's remaining "real external IdP" gap. Also gitignores docker-compose.override.yml, since that's where such real credentials belong for local testing -- never committed. |
||
|
jcoffey-dev
|
abeee0076b |
Build and browser-verify the tenant-picker frontend page
web/src/routes/select-tenant now calls enterprise-auth's existing
GET /auth/memberships / POST /auth/select-tenant protocol (built earlier
this phase, previously called only from Go tests) via
fetch(..., {credentials: 'include'}) -- new listMemberships/selectTenant
functions in $lib/api.ts, using a dedicated request helper that reads
plain-text error bodies (loginhandler's http.Error responses), unlike
every other request helper in that file which expects JSON.
Credentialed cross-origin fetch needed CORS enterprise-auth didn't have:
api/httpserver.WithCORS's wildcard-friendly default can't be combined
with a credentialed request at all (browsers refuse to honor
Access-Control-Allow-Origin: "*" on one) -- added WithCredentialedCORS
(literal origin, Access-Control-Allow-Credentials: true) alongside it,
wired into enterprise-auth via a new CORS_ALLOWED_ORIGIN config var
defaulting to POST_LOGIN_REDIRECT_URL (web's own origin, the same
default pattern SELECT_TENANT_REDIRECT_URL already used).
adapter-static's route crawler doesn't discover a page nothing links to
(this one is only ever reached via enterprise-auth's redirect) -- fixed
with select-tenant/+page.ts's `export const prerender = true`, the same
declaration every other route already has.
Genuinely verified in a real browser in this environment, not just
type-checked: a throwaway Node server standing in for enterprise-auth's
exact wire contract (including its plain-text error bodies), driven
through the full flow via mcp__claude-in-chrome -- cross-origin
pending-login cookie set, credentialed preflight + GET/POST round trip,
a real click choosing a tenant, the post-selection redirect, and the
missing/expired-cookie error path rendering the backend's actual
message. No Docker or live Postgres/IdP needed, since the point was
exercising web's own fetch/CORS/cookie wiring, not enterprise-auth's
internals (already covered by loginhandler's own tests).
This closes the tenant-picker as the last named gap in Phase 4. What's
left is the already-disclosed live-verification caveat shared by every
Postgres/ClickHouse-backed piece and both SSO protocols: none of this
has run against a real database, external IdP, or multi-container
deployment in this environment.
|
||
|
jcoffey-dev
|
3eb0f4c589 |
Phase 4: SSO scaffolding, RBAC enforcement, tenant-scoped dashboards, audit logging, K8s deployment
RBAC (api/internal/authz) is live on /query and /dashboards, backed by a new enterprise/ module (session issuance, audit logging, RBAC storage, OIDC/SAML protocol wiring) that core never imports -- only calls over HTTP. Found and fixed a real cross-tenant vulnerability in dashboards (no tenant_id filtering at all) while writing the threat model doc. Two things are explicitly NOT done, documented rather than hidden: tenant isolation for log data itself (/query still shares one ClickHouse connection and Tantivy index across every tenant -- RBAC controls who can query, not what a query can see), and human SSO login (protocol wiring exists, no HTTP handler calls it yet). See docs/security/threat-model.md and docs/phase-4-runbook.md. Also adds deploy/ (Go Operator + Helm chart, validated offline only -- no cluster was reachable in this environment). |
||
|
jcoffey-dev
|
9435115ab7 |
Phase 3: dashboards and alerting
Saved, shareable multi-panel dashboards (table/line/bar/single-stat panels via gridstack + uPlot, global + per-panel time range, JSON export/import) and threshold/absence alert rules with an ok/pending/firing evaluator and webhook/Slack/PagerDuty delivery. - New /metadata component: Postgres control-plane store for dashboards, panels, notification targets, alert rules/state, and delivery log -- see docs/phase-3-dashboard-design.md for why ClickHouse's MergeTree family isn't a fit for this access pattern (needs real row-level locking and read-your-writes consistency). - api/internal/dashboards: dashboard/panel CRUD, pure -- panel query execution stays client-side, reusing the existing /query endpoint. - New /alerting service: rule/target CRUD, a ticker-driven evaluator (claim-then-evaluate concurrency control, transactional-outbox delivery, query errors and threshold zero-rows never coerced into a false transition) and webhook/Slack/PagerDuty delivery with retry/backoff. See docs/phase-3-alerting-design.md for the full state-machine design and the four correctness properties it implements. - web: /dashboards and /alerts UIs; cli: sentryctl dashboards/alerts list/get/apply, seeding a future Terraform provider's JSON contract. - hack/alert-load-test: 500 rules against real ClickHouse data, real measured results in docs/phase-3-runbook.md. Five real bugs found by actually running this against a live stack (documented in the runbook, not just fixed silently): a latent Phase 2 bug where ClickHouse rejected the timestamp format used for earliest=/latest= queries; a "now" literal token injected into query text; a GridStack/uPlot layout-timing race; JS's Date.parse being too lenient to use as a timestamp-detection heuristic; a rule's "enabled" field silently defaulting to false when omitted; and the evaluator's claim-batch-size and worker-pool-concurrency defaulting to the same value, causing 500 concurrently-due rules to take 125s to cycle through instead of the configured 60s. |
||
|
jcoffey-dev
|
fb5049a747 |
Phase 2: unified query language spanning ClickHouse and Tantivy
Replaces the separate SQL-only /query and text-only /search endpoints with one pipe-syntax query language (plus raw SQL escape hatch) that compiles to a single IR and execution plan across both backends, so a query like `message:"connection refused" | stats count by host` runs as one request instead of two disjoint tools. - api/internal/querylang: lexer -> ast -> parser -> ir -> planner -> executor, each layer independently tested. - Execution generalizes Phase 1's proven Tantivy-prefilter pattern into a 4-way routing table (pure ClickHouse / text-only / text + aggregation / raw SQL passthrough). - Unified web query page and `sentryctl query`, both hitting the same POST /query endpoint. - Benchmarked against a real 1,022,000-row dataset (hack/benchmark-fixture); caught and fixed a real bug where the Tantivy prefilter cap (10,000) produced an IN-clause exceeding ClickHouse's default max_query_size -- lowered to 5,000, documented in docs/query-language-design.md and docs/phase-2-runbook.md. - docs/query-language-reference.md: customer-facing syntax reference. |
||
|
jcoffey-dev
|
cd8aa290ca |
Phase 1: Windows log collection + full-text search
Extends the agent, ingest, storage, api, and web with Windows Event Log/ETW sourcing and Tantivy-backed free-text search, per the approved Phase 1 plan. - CLAUDE.md: materialized on disk (never existed as a file before) with a new Phase 1 "done looks like" section. - agent: Windows Event Log (EvtSubscribe) and ETW sources, Windows service wrapper (install/uninstall/run-service), both feature- and target_os-gated so Linux builds/tests/clippy stay unaffected. Also fixed two pre-existing Phase 0 clippy gaps (dead-code on default-features-only builds, a type-inference edge case) found while testing every feature combination properly for the first time. UNVERIFIED on real Windows -- no Windows toolchain existed anywhere in the build environment; flagged prominently in three places. - proto/ingest: new record_id field, assigned once server-side in ingest's gRPC front end so ClickHouse and Tantivy agree on the same ID for the same record. - storage: record_id column + bloom filter index, verified against a live ClickHouse. - search: new service, Tantivy index, rskafka consumer as an independent second consumer group on the same Redpanda topic ingest already reads. - api/web: new /search endpoint and page, sharing the query page's result-table shape and component. - hack/windows-fixture: sends realistic Windows-shaped data straight to ingest, so the pipeline's handling of it is verifiable without a Windows host. Verified end-to-end on the live docker-compose stack: the same record_id comes back from both /query and /search for the same log line, including for windows-fixture's synthetic Windows Event Log data. Real bugs found and fixed along the way: api/Dockerfile missing proto/ in its build context, search's logs being completely silent (RUST_LOG gap), and search/target/ missing from .gitignore/.dockerignore. |
||
|
jcoffey-dev
|
b6b092c912 |
Scaffold Phase 0: agent -> Redpanda -> ingest -> ClickHouse -> api -> web
End-to-end log pipeline for Linux hosts, per /docs/architecture.md: - proto: shared gRPC contract (agent <-> ingest), Go bindings checked in - agent: Rust, musl-targeted, journald/file sourcing, RFC5424 parser, mTLS gRPC client, no required config for the common case - ingest: Go, single binary with --mode server|consumer|all; gRPC front end forwards to Redpanda unchanged, consumer normalizes and batch-writes to ClickHouse with at-least-once delivery - storage: ClickHouse schema + a plain SQL-file migration runner - api: minimal SELECT-only query endpoint, plain REST (not gRPC+gateway yet -- see api/README.md) - web: SvelteKit static SPA, one query page - transport: Redpanda compose + topic provisioning - cli: sentryctl ping stub - hack/dev-certs: throwaway CA + cert generation for local mTLS - root docker-compose.yml + docs/phase-0-runbook.md tie it together Not yet run end-to-end against real Docker/ClickHouse/Redpanda -- see the runbook's caveats section before relying on this working as-is. |