coffey-labs/cairnobs
1
0
Fork 0
Code Issues Pull Requests 1 Actions Packages Projects Releases Wiki Activity
191 Commits 2 Branches 0 Tags
main
Commit Graph
38 Commits
Author SHA1 Message Date
dependabot[bot] 6a05e9ce5c Bump the go-minor-and-patch group across 4 directories with 2 updates 
Bumps the go-minor-and-patch group with 1 update in the /alerting directory: [github.com/jackc/pgx/v5](https://github.com/jackc/pgx).
Bumps the go-minor-and-patch group with 2 updates in the /api directory: [github.com/jackc/pgx/v5](https://github.com/jackc/pgx) and [golang.org/x/crypto](https://github.com/golang/crypto).
Bumps the go-minor-and-patch group with 1 update in the /enterprise directory: [github.com/jackc/pgx/v5](https://github.com/jackc/pgx).
Bumps the go-minor-and-patch group with 1 update in the /ingest directory: [github.com/jackc/pgx/v5](https://github.com/jackc/pgx).


Updates `github.com/jackc/pgx/v5` from 5.10.0 to 5.11.0
- [Release notes](https://github.com/jackc/pgx/releases)
- [Changelog](https://github.com/jackc/pgx/blob/master/CHANGELOG.md)
- [Commits](https://github.com/jackc/pgx/compare/v5.10.0...v5.11.0)

Updates `github.com/jackc/pgx/v5` from 5.10.0 to 5.11.0
- [Release notes](https://github.com/jackc/pgx/releases)
- [Changelog](https://github.com/jackc/pgx/blob/master/CHANGELOG.md)
- [Commits](https://github.com/jackc/pgx/compare/v5.10.0...v5.11.0)

Updates `golang.org/x/crypto` from 0.56.0 to 0.57.0
- [Commits](https://github.com/golang/crypto/compare/v0.56.0...v0.57.0)

Updates `github.com/jackc/pgx/v5` from 5.10.0 to 5.11.0
- [Release notes](https://github.com/jackc/pgx/releases)
- [Changelog](https://github.com/jackc/pgx/blob/master/CHANGELOG.md)
- [Commits](https://github.com/jackc/pgx/compare/v5.10.0...v5.11.0)

Updates `github.com/jackc/pgx/v5` from 5.10.0 to 5.11.0
- [Release notes](https://github.com/jackc/pgx/releases)
- [Changelog](https://github.com/jackc/pgx/blob/master/CHANGELOG.md)
- [Commits](https://github.com/jackc/pgx/compare/v5.10.0...v5.11.0)

---
updated-dependencies:
- dependency-name: github.com/jackc/pgx/v5
  dependency-version: 5.11.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-minor-and-patch
- dependency-name: github.com/jackc/pgx/v5
  dependency-version: 5.11.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-minor-and-patch
- dependency-name: golang.org/x/crypto
  dependency-version: 0.57.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-minor-and-patch
- dependency-name: github.com/jackc/pgx/v5
  dependency-version: 5.11.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-minor-and-patch
- dependency-name: github.com/jackc/pgx/v5
  dependency-version: 5.11.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-minor-and-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
 2026-09-15 09:12:19 +00:00
dependabot[bot] 82cfe1fa0a Bump the go-minor-and-patch group across 5 directories with 9 updates 
Bumps the go-minor-and-patch group with 1 update in the /alerting directory: [golang.org/x/sync](https://github.com/golang/sync).
Bumps the go-minor-and-patch group with 1 update in the /api directory: [golang.org/x/crypto](https://github.com/golang/crypto).
Bumps the go-minor-and-patch group with 3 updates in the /deploy/operator directory: [k8s.io/apimachinery](https://github.com/kubernetes/apimachinery), [k8s.io/client-go](https://github.com/kubernetes/client-go) and [sigs.k8s.io/controller-runtime](https://github.com/kubernetes-sigs/controller-runtime).
Bumps the go-minor-and-patch group with 6 updates in the /enterprise directory:

| Package | From | To |
| --- | --- | --- |
| [golang.org/x/sync](https://github.com/golang/sync) | `0.22.0` | `0.23.0` |
| [k8s.io/apimachinery](https://github.com/kubernetes/apimachinery) | `0.31.0` | `0.37.0` |
| [k8s.io/client-go](https://github.com/kubernetes/client-go) | `0.31.0` | `0.37.0` |
| [github.com/coreos/go-oidc/v3](https://github.com/coreos/go-oidc) | `3.20.0` | `3.21.0` |
| [github.com/go-jose/go-jose/v4](https://github.com/go-jose/go-jose) | `4.1.4` | `4.1.5` |
| [golang.org/x/oauth2](https://github.com/golang/oauth2) | `0.36.0` | `0.37.0` |

Bumps the go-minor-and-patch group with 1 update in the /ingest directory: [golang.org/x/sync](https://github.com/golang/sync).


Updates `golang.org/x/sync` from 0.22.0 to 0.23.0
- [Commits](https://github.com/golang/sync/compare/v0.22.0...v0.23.0)

Updates `golang.org/x/sync` from 0.22.0 to 0.23.0
- [Commits](https://github.com/golang/sync/compare/v0.22.0...v0.23.0)

Updates `golang.org/x/sync` from 0.22.0 to 0.23.0
- [Commits](https://github.com/golang/sync/compare/v0.22.0...v0.23.0)

Updates `golang.org/x/crypto` from 0.55.0 to 0.56.0
- [Commits](https://github.com/golang/crypto/compare/v0.55.0...v0.56.0)

Updates `k8s.io/apimachinery` from 0.31.0 to 0.37.0
- [Commits](https://github.com/kubernetes/apimachinery/compare/v0.31.0...v0.37.0)

Updates `k8s.io/client-go` from 0.31.0 to 0.37.0
- [Changelog](https://github.com/kubernetes/client-go/blob/master/CHANGELOG.md)
- [Commits](https://github.com/kubernetes/client-go/compare/v0.31.0...v0.37.0)

Updates `sigs.k8s.io/controller-runtime` from 0.19.3 to 0.25.0
- [Release notes](https://github.com/kubernetes-sigs/controller-runtime/releases)
- [Changelog](https://github.com/kubernetes-sigs/controller-runtime/blob/main/RELEASE.md)
- [Commits](https://github.com/kubernetes-sigs/controller-runtime/compare/v0.19.3...v0.25.0)

Updates `k8s.io/api` from 0.31.0 to 0.37.0
- [Commits](https://github.com/kubernetes/api/compare/v0.31.0...v0.37.0)

Updates `k8s.io/apimachinery` from 0.31.0 to 0.37.0
- [Commits](https://github.com/kubernetes/apimachinery/compare/v0.31.0...v0.37.0)

Updates `k8s.io/client-go` from 0.31.0 to 0.37.0
- [Changelog](https://github.com/kubernetes/client-go/blob/master/CHANGELOG.md)
- [Commits](https://github.com/kubernetes/client-go/compare/v0.31.0...v0.37.0)

Updates `golang.org/x/sync` from 0.22.0 to 0.23.0
- [Commits](https://github.com/golang/sync/compare/v0.22.0...v0.23.0)

Updates `k8s.io/apimachinery` from 0.31.0 to 0.37.0
- [Commits](https://github.com/kubernetes/apimachinery/compare/v0.31.0...v0.37.0)

Updates `k8s.io/client-go` from 0.31.0 to 0.37.0
- [Changelog](https://github.com/kubernetes/client-go/blob/master/CHANGELOG.md)
- [Commits](https://github.com/kubernetes/client-go/compare/v0.31.0...v0.37.0)

Updates `github.com/coreos/go-oidc/v3` from 3.20.0 to 3.21.0
- [Release notes](https://github.com/coreos/go-oidc/releases)
- [Commits](https://github.com/coreos/go-oidc/compare/v3.20.0...v3.21.0)

Updates `github.com/go-jose/go-jose/v4` from 4.1.4 to 4.1.5
- [Release notes](https://github.com/go-jose/go-jose/releases)
- [Commits](https://github.com/go-jose/go-jose/compare/v4.1.4...v4.1.5)

Updates `golang.org/x/oauth2` from 0.36.0 to 0.37.0
- [Commits](https://github.com/golang/oauth2/compare/v0.36.0...v0.37.0)

Updates `golang.org/x/sync` from 0.22.0 to 0.23.0
- [Commits](https://github.com/golang/sync/compare/v0.22.0...v0.23.0)

Updates `k8s.io/api` from 0.31.0 to 0.37.0
- [Commits](https://github.com/kubernetes/api/compare/v0.31.0...v0.37.0)

Updates `k8s.io/apimachinery` from 0.31.0 to 0.37.0
- [Commits](https://github.com/kubernetes/apimachinery/compare/v0.31.0...v0.37.0)

Updates `k8s.io/client-go` from 0.31.0 to 0.37.0
- [Changelog](https://github.com/kubernetes/client-go/blob/master/CHANGELOG.md)
- [Commits](https://github.com/kubernetes/client-go/compare/v0.31.0...v0.37.0)

Updates `golang.org/x/sync` from 0.22.0 to 0.23.0
- [Commits](https://github.com/golang/sync/compare/v0.22.0...v0.23.0)

Updates `golang.org/x/sync` from 0.22.0 to 0.23.0
- [Commits](https://github.com/golang/sync/compare/v0.22.0...v0.23.0)

Updates `golang.org/x/sync` from 0.22.0 to 0.23.0
- [Commits](https://github.com/golang/sync/compare/v0.22.0...v0.23.0)

Updates `golang.org/x/sync` from 0.22.0 to 0.23.0
- [Commits](https://github.com/golang/sync/compare/v0.22.0...v0.23.0)

---
updated-dependencies:
- dependency-name: golang.org/x/sync
  dependency-version: 0.23.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-minor-and-patch
- dependency-name: golang.org/x/sync
  dependency-version: 0.23.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-minor-and-patch
- dependency-name: golang.org/x/sync
  dependency-version: 0.23.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-minor-and-patch
- dependency-name: golang.org/x/crypto
  dependency-version: 0.56.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-minor-and-patch
- dependency-name: k8s.io/apimachinery
  dependency-version: 0.37.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-minor-and-patch
- dependency-name: k8s.io/client-go
  dependency-version: 0.37.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-minor-and-patch
- dependency-name: sigs.k8s.io/controller-runtime
  dependency-version: 0.25.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-minor-and-patch
- dependency-name: k8s.io/api
  dependency-version: 0.37.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-minor-and-patch
- dependency-name: k8s.io/apimachinery
  dependency-version: 0.37.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-minor-and-patch
- dependency-name: k8s.io/client-go
  dependency-version: 0.37.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-minor-and-patch
- dependency-name: golang.org/x/sync
  dependency-version: 0.23.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-minor-and-patch
- dependency-name: k8s.io/apimachinery
  dependency-version: 0.37.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-minor-and-patch
- dependency-name: k8s.io/client-go
  dependency-version: 0.37.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-minor-and-patch
- dependency-name: github.com/coreos/go-oidc/v3
  dependency-version: 3.21.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-minor-and-patch
- dependency-name: github.com/go-jose/go-jose/v4
  dependency-version: 4.1.5
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: go-minor-and-patch
- dependency-name: golang.org/x/oauth2
  dependency-version: 0.37.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-minor-and-patch
- dependency-name: golang.org/x/sync
  dependency-version: 0.23.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-minor-and-patch
- dependency-name: k8s.io/api
  dependency-version: 0.37.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-minor-and-patch
- dependency-name: k8s.io/apimachinery
  dependency-version: 0.37.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-minor-and-patch
- dependency-name: k8s.io/client-go
  dependency-version: 0.37.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-minor-and-patch
- dependency-name: golang.org/x/sync
  dependency-version: 0.23.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-minor-and-patch
- dependency-name: golang.org/x/sync
  dependency-version: 0.23.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-minor-and-patch
- dependency-name: golang.org/x/sync
  dependency-version: 0.23.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-minor-and-patch
- dependency-name: golang.org/x/sync
  dependency-version: 0.23.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-minor-and-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
 2026-09-10 17:00:23 +00:00
jcoffey-dev c60028aad1 Move the Go toolchain pins to 1.26, in CI and in every image 
Two Dependabot PRs are stuck behind the same number.

#35 raises the go directive to 1.26.0 in six modules, because
golang.org/x/crypto v0.56.0 requires it -- x/crypto tracks the two most
recent Go releases and 0.56 dropped 1.25. A module that says 1.26 cannot
be built by the 1.25 this repository pins in two places, so that PR
fails every Go job.

#29 raises actions/setup-go to v7, which sets GOTOOLCHAIN=local. With
that set, `go install golang.org/x/vuln/cmd/govulncheck@latest` cannot
quietly fetch a newer toolchain, and stops with

  golang.org/x/[email protected] requires go >= 1.26.0 (running go 1.25.14)

Under setup-go v5 the same install succeeded by downloading 1.26 behind
our backs, which is its own reason to be on 1.26 deliberately instead.

So: security-scan's go-version and all eight Dockerfiles move together,
1.25 -> 1.26. Nothing else needs to. A newer toolchain builds an older
directive happily, so this stands on its own before #35 lands, and the
go.mod files stay where they are here.

Checked by building rather than by reading: the api and ingest images
both build on golang:1.26-alpine, and api, ingest and enterprise still
`go build ./...` clean against their existing 1.25 directives.
 2026-09-10 09:54:45 -07:00
jcoffey-dev b7f99b49f2 Take grpc to 1.83.2 across the nine modules that carry it 
GHSA-2v4p-qf9q-27wj is a panic in gRPC-Go's xDS routing interceptor: a
request arriving with neither `:authority` nor `Host` indexes an empty
slice, the per-RPC goroutine does not recover, and the process dies.
High, and nine alerts, because nine go.mod files pin the same version --
eight directly, terraform indirectly.

Nothing here was reachable. The interceptor is installed by
`xds.NewGRPCServer`, which this repo never calls: the one production
server is `grpc.NewServer(grpc.Creds(...))` in ingest/internal/grpcserver
and the only other is a plain one in a searchclient test. That is also
why security-scan has been green throughout -- govulncheck reports on
reachability and found nothing on 1.83.1, while Dependabot reports on
version ranges and found nine. Both were right.

Taken anyway: it is a patch release, and the next advisory in this
dependency may well land somewhere we do reach.

`go mod tidy` carried the indirect requirements grpc 1.83.2 asks for --
x/net, x/text, x/sys and friends. No CI job builds or tests Go here, so
all nine modules were built locally and api, ingest and enterprise
tested with -count=1, since a cached pass would not have exercised the
new version.

The dependabot.yml is the other half. There was no config, so nothing
opened a PR against any of this. Go majors stay out of the group, being
import path changes rather than bumps.
 2026-09-10 09:01:27 -07:00
jcoffey-dev 200f801e2c Clear the Dependabot findings 
Dependabot alerts were switched on for this repo today and reported 12 open
findings. Ten are fixed here; the other two are addressed below.

gRPC 1.83.0 -> 1.83.1, in all nine modules that require it. This is
GHSA-vp52-pcj8-j9qc / CVE-2026-84304, heap memory exhaustion via HTTP/2 DATA
frame fragmentation, affecting <= 1.83.0. It matters more than the version
delta suggests: ingest/ is a gRPC listener deliberately exposed to the internet
on :4317, so a remote OOM is reachable. mTLS narrows that to holders of a
client certificate, which is why this was not an emergency, but the fix is one
patch release away and there is no reason to carry it.

golang.org/x/oauth2 0.21.0 -> 0.27.0 in deploy/operator, an indirect
dependency (GHSA-6v2p-p543-phr9). enterprise/ was already past it at 0.36.0.

npm cookie 0.6.0 -> 0.7.2, via an overrides entry rather than a dependency
bump. @sveltejs/kit requires ^0.6.0 and still does at 2.70.3, the latest
release, so there is no version of kit that resolves this on its own -- an
override is the only route that does not involve waiting on upstream.

Three incidental changes came out of `go mod tidy` and are not mine:
genproto/googleapis/rpc moved forward as a transitive of the new grpc; pgx/v5
was reclassified from indirect to direct in enterprise/, which is simply
correct, since audit.go and cmd/enterprise-auth import it; and the proto
replace directive shuffled between require blocks at the same version.

The twelfth finding, lru (GHSA-rhfx-m35p-ff5j), is not fixed and is not
fixable here -- see the note in the pull request. It is CVSS 0, a Stacked
Borrows soundness issue in IterMut, and reaching a patched version means
tantivy 0.22 -> 0.26, which is a search engine migration rather than a
dependency bump.

Verified: all ten Go modules build, 40 test packages pass, the web app builds
and svelte-check reports 0 errors across 288 files.
 2026-09-03 11:12:35 -07:00
jcoffey-dev f756a9d4f6 Rename the project spec and update every reference to it 
The charter file carried a tool-specific name while being the repository's own
document: mission, non-negotiable constraints, the pinned stack, repo
conventions and phase status, cited as authority by thirty files across the
agent, api, deploy, docs, search and terraform trees.

PROJECT-SPEC.md says what it is. All 42 references are updated in the same
commit, including the relative link in docs/status.md, so nothing points at a
filename that no longer exists.
 2026-08-28 15:57:12 -07:00
jcoffey-dev 7a86008062 Complete the low-risk half of the Sentry -> Cairn OBS rebrand 
Sweeps the references that carry no runtime coupling, and fixes one that
turned out to be a real bug rather than stale branding.

Docker network: sentry_default -> cairnobs_default across 23 runbook and
test-header `docker run` commands. Compose derives the network from the
directory name, so this lands together with renaming the working copy to
cairnobs/ -- the two are only correct as one change.

Stale references corrected: four Dockerfile "repo root (sentry/)"
headers; .env pointing at the long-renamed deploy/helm/sentry/ chart;
five Helm comments describing the topic as sentry.logs.raw when all four
code paths have defaulted to cairnobs.logs.raw for some time; an
absolute /home/john/Projects/sentry/ path in the operator's package doc,
now repo-relative; the hand-written Tenant CRD description in both of
its identical copies, whose Go source already said Cairn OBS.

Migration 0043 repoints the default tenant's data source. 0026 seeded it
with ('sentry', '/var/lib/sentry-search') to match what
api/internal/config then defaulted to; the rebrand later moved those
defaults to "cairnobs" and /var/lib/cairnobs-search without moving the
already-applied row, leaving the default tenant naming a ClickHouse
database nothing writes to. Scoped to the exact stale values so it is a
no-op on any deployment that set them deliberately. 0026's comment is
annotated as superseded; its applied SQL is untouched.

Deliberately not included: the gRPC wire packages (sentry.logs.v1,
sentry.agent.v1) and proto/sentry/ import paths, which cannot change
without a lockstep agent/server upgrade; the Helm chart's
sentry_metadata database and sentry role, which need a real Postgres
migration on existing deployments; and the compliance audit records in
docs/compliance/, which are a dated historical record.

go build, go vet, and go test pass for ingest and deploy/operator.
 2026-08-22 18:39:25 -07:00
jcoffey-dev c920e0f2c4 Finish the Cairn OBS rename through services, docs, and assets 
The rename commit before this one covered module paths and the obvious
user-facing strings; this is the rest of it -- the places where "sentry"
was a default value, a filename, or a picture rather than a word in a
sentence.

Defaults that changed: CLICKHOUSE_DATABASE (sentry -> cairnobs),
POSTGRES_DATABASE (sentry_metadata -> cairnobs_metadata), and
POSTGRES_USERNAME (sentry -> cairnobs), across api/alerting/ingest and
the enterprise binaries, plus the compose files and migrate scripts that
create those objects. These are *defaults*, so a deployment that sets
them explicitly is unaffected -- but any deployment relying on the old
defaults must have its environment updated before it picks this up, or
it will come up pointing at a database that doesn't exist.

Also: the light-mode logo variants (the dark ones existed alone, so the
landing page and sidebar rendered a dark mark on a light background),
regenerated favicons, and the docs/README/threat-model prose that still
said Sentry.
 2026-08-22 16:12:08 -07:00
jcoffey-dev 13cf9a30cb Rebrand: Sentry -> Cairn OBS 
Full rebrand across cosmetic branding, code identifiers, and
infrastructure/data-plane naming, using the supplied Cairn OBS logo
package. Cosmetic: favicon/logo swap (also closes a stale license-audit
finding -- the old favicon was SvelteKit's unreplaced scaffold logo),
new centered welcome landing page, larger/legible sidebar logo, page
titles, CLAUDE.md/README/docs prose.

Code identifiers: Go module path github.com/sentry/sentry ->
github.com/cairnobs/cairnobs across all 13 modules and ~91 files (protoc
regenerated); Rust crates sentry-agent/sentry-parser/sentry-search ->
cairnobs-*; CLI sentryctl -> cairnobsctl; Terraform provider fully
renamed (sentry_dashboard etc. -> cairnobs_dashboard, provider type,
env vars); every session/auth cookie name; agent config paths and
Windows service identity.

Deliberately preserved: the gRPC wire protocol's protobuf packages
(sentry.logs.v1, sentry.agent.v1) and their Go import directory
(proto/sentry/...) -- renaming the wire-level package would break every
currently-deployed agent binary (confirmed two real hosts, including
mail.inbuxa.com, are actively streaming through this exact contract)
until rebuilt and redeployed in lockstep with an ingest cutover. Only
the Go module path wrapping the generated code changes.

Infrastructure: every docker-compose container name (root and three
component-level compose files); the Helm chart (directory, Chart.yaml,
named-template helpers, all templates, values.yaml image repos);
Kubernetes Operator (CRD group sentry.io -> cairnobs.io, both CRD YAML
files, Go identifiers, RBAC markers); the coupled enterprise/tenantcrd
package. Caught and fixed real path-coupling bugs along the way: the
Helm chart's search/ingest volume mounts and the dev-only-credential
detection constant vs. docker-compose.yml's literal values had to move
together or a security warning would have silently stopped firing.

Data plane: Postgres database sentry_metadata -> cairnobs_metadata and
role sentry -> cairnobs; ClickHouse database sentry -> cairnobs; Kafka
topic sentry.logs.raw -> cairnobs.logs.raw and its consumer groups.
Source-level defaults, docker-compose.yml, and every migrate.sh/
provision script default updated together; already-applied migration
files left untouched per this repo's immutable-migration convention.

Verified at every layer: all 13 Go modules build/vet/test clean, both
Rust workspaces (agent, search) build/clippy/test clean, npm run check/
build clean, docker compose config validates on all four compose files.
Live-verified against a real docker stack multiple times through this
work, including a final fresh-volume run confirming the actual renamed
Postgres database/role, ClickHouse database, and Kafka topic all work
end to end with a real login and query, zero console errors.
 2026-08-21 20:53:32 -07:00
jcoffey-dev 4b5dae5879 Add local login, agent extra log paths, IPv4/IPv6 metrics; remediate security audit findings 
This is a large squashed commit covering two batches of prior uncommitted
work plus a full security-audit remediation pass, kept together because
go.mod/go.sum and several shared files (main.go, handler.go) were touched
by both and splitting risked non-building intermediate commits.

Features (built earlier, previously uncommitted):
- Local username/password login for single-tenant deployments with no
  SSO configured (api/localauth, alerting/internal/sessioncheck,
  sentryctl users, web/src/routes/login, metadata migrations 0040/0041).
- Remotely-editable additional log file paths for agents, on top of
  their existing primary source (api/agents, agent/sentry-agent
  extra-file-path diffing, web agent config UI).
- IPv4/IPv6 addresses reported alongside other host system metrics.

Security audit remediation (this pass, all live-verified in production):
- Critical: block ClickHouse SSRF table functions (url/remote/file/s3/...)
  in the raw-SQL query escape hatch.
- High: deny sensitive paths and require Admin to add agent
  extra_file_paths (Editor could previously point an agent at /etc/shadow
  or an SSH key); alerting webhook targets now validate against
  internal/metadata/loopback addresses, both at creation and send time;
  alerting's session middleware now enforces an Editor+ floor on
  mutating requests instead of "any authenticated session"; bumped
  goxmldsig to close a SAML signature-verification bypass (GO-2026-4753).
- Medium: per-IP login rate limiting; security response headers
  (HSTS/CSP/nosniff/X-Frame-Options/Referrer-Policy/Permissions-Policy)
  on web/nginx.conf; a DevCredentialWarnings check in every Go service's
  config loader, logging loudly at startup if a deployment is still on
  docker-compose.yml's literal dev-only credentials; dependency bumps
  (golang.org/x/text, grpc, x/net, quick-xml, h2) across every affected
  Go module and both Rust crates, including a previously-uncovered x/net
  vulnerability in deploy/operator; a new security-scan.yml CI workflow
  running cargo-deny/govulncheck/npm-audit, mirroring the existing
  license-compliance.yml matrix shape.
- Low: removed sentryctl's plaintext --password flag (shell
  history/`ps` exposure) in favor of stdin and a --password-stdin flag
  for reset-password's optional specific-password path; a dummy bcrypt
  comparison closes a login response-time username-enumeration
  side-channel.
 2026-08-18 23:53:20 -07:00
jcoffey-dev 93c160ec51 Add agent restart lifecycle command 
Extends the existing CheckIn RPC with a one-shot AgentCommand
(restart only -- stop/uninstall need real per-platform OS
service-manager integration and stay deliberately out of scope),
delivered at-most-once: cleared the instant it's handed to the agent
in a response, since a restarting agent's process is gone before it
could ever confirm receipt. On restart, the agent flushes whatever's
buffered, aborts its source task, and exits cleanly, relying entirely
on the host's own service manager to bring it back up.

Issuing a command is gated at RoleAdmin (stricter than config
editing's RoleEditor) and logged into the same audit_log table Phase
7's AI interactions use, via a new agent_command event type.

A real bug was found and fixed during live verification: the first
implementation tried to atomically read-and-clear pending_command in
a single INSERT...ON CONFLICT statement using a sibling CTE
referenced only from RETURNING, on the assumption that Postgres
evaluates every part of a WITH query against one pre-statement
snapshot. That's wrong specifically for FOR UPDATE, which always
reads the latest row version including one written earlier in the
same statement -- confirmed empirically (a restart command was
always coming back empty even when genuinely pending, so the agent
never received it). Fixed by splitting into two real, ordered
statements inside one explicit transaction.

See /docs/agent-management-design.md's "Lifecycle commands" section.
 2026-08-16 20:30:07 -07:00
jcoffey-dev 4f0da1ae5e Add agent inventory, management, and remote config 
Extends the heartbeat mechanism with a second gRPC service on the same
mTLS channel (AgentControl.CheckIn, agent-initiated on the existing
heartbeat ticker -- still push-only, no inbound port on any agent) so
an agent reports its running config and can pick up an operator-set
override. A new web UI section (/agents) lists every agent that's
checked in, shows its reported config, and lets an operator edit a
narrow, deliberately-scoped subset remotely: batch/heartbeat tuning,
and (journald sources only) the unit filter.

TLS material and the ingest endpoint are never reportable or remotely
editable, by proto shape rather than a validation rule -- a bad or
malicious edit there could permanently strand an agent or redirect
where its logs go, unlike every other editable field, which only
degrades behavior.

An override lives only in the agent's memory (agent.toml is never
rewritten) and re-syncs on the agent's own schedule; changing the
journald filter aborts and respawns the source task since there's no
other way to change what's being tailed. Building the hot-reload path
surfaced a real, independent, pre-existing bug: shutdown was using
poll_timeout(), which only drains once flush_interval has elapsed,
silently dropping anything buffered more recently on every graceful
shutdown that landed between flushes -- fixed with a new unconditional
Batcher::flush_all(), now used at both shutdown and hot-reload.

Verified live end-to-end against a real stack: an edited heartbeat
interval changed a running agent's actual send cadence within one
check-in cycle (confirmed by the real timestamps landing in
ClickHouse), and an edited journald filter triggered a real source
restart, both reflected back in the next reported-config snapshot.

See /docs/agent-management-design.md.
 2026-08-16 18:08:51 -07:00
jcoffey-dev 7d316f92db Phase 7: AI-assisted query authoring (autocomplete, explain, fix, optimize, NL translation) 
Adds a self-hosted (Ollama, qwen2.5-coder) model provider abstraction
with a pluggable opt-in cloud adapter, schema grounding, and a shared
cost/safety guard every AI-suggested query is assessed against --
compiling to and executing through the same unchanged Phase 2 IR/
compiler and Phase 4 tenant scoping as a hand-written query, no
parallel execution path.

Track A (built into the query bar): inline ghost-text autocomplete,
"Explain this query", "Fix this query" with a diff view, and a
rule-based "Optimize" suggestion. Track B: natural-language-to-query
translation, always a separate review step from execution, with
`sentryctl query --nl` requiring explicit confirmation to run.
Every accepted/dismissed translate-fix-optimize interaction is logged
into the same append-only audit_log table Phase 4 built.

Two real product bugs were found and fixed via live browser
verification (a Svelte effect re-running on every keystroke that
silently cancelled the ghost-text debounce; a ghost-text widget
positioned at document offset 0 instead of the cursor), and a real
costguard logic bug (unbounded-aggregation vs. raw-row) was caught by
its own test suite. New integration tests wire a real Ollama client
through the real HTTP handler against a mock server matching Ollama's
wire contract (hack/mock-ollama), keeping model-quality verification
out of CI as a disclosed, periodic human-run check instead.

See /docs/phase-7-ai-design.md and /docs/phase-7-runbook.md.
 2026-08-16 18:06:27 -07:00
jcoffey-dev 661568085e Phase 6: license-compliance audit and enterprise/ relicensing to AGPLv3 
Full dependency inventory across Rust/Go/npm plus Docker base images
and vendored assets (776 rows, 502 unique deps), classified against
AGPLv3 compatibility with real citations rather than assumptions.
enterprise/ relicensed from its commercial-license stub to AGPLv3,
matching core -- the one real flag (Redpanda's BSL 1.1) was evaluated
against primary sources and accepted as-is rather than triggering a
broker swap. CI enforcement wired up (.github/workflows/license-
compliance.yml, this repo's first CI workflow), a root LICENSE file
added, and every doc/comment referencing the old commercial-license
boundary updated to describe it as architectural only.

See /docs/compliance/ for the full report, inventory, and policy.
 2026-08-16 18:03:32 -07:00
jcoffey-dev f5ca09f686 Honor X-Forwarded-Proto for cookie Secure, not just r.TLS 
Every auth cookie loginhandler.go sets (OIDC state, SAML request,
pending-login, session) decided Secure from r.TLS != nil alone --
correct only if enterprise-auth terminates TLS itself, which it never
does (it's a plain http.Server, same as every other service here). In
any real deployment, TLS is terminated at a reverse proxy/ingress in
front of it, so r.TLS is nil at this process even over a genuinely
HTTPS client connection.

Found live: SAML's request-tracking cookie is SameSite=None (required,
since the ACS POST is cross-site from the IdP's origin), which the
cookie spec requires to be paired with Secure. Behind a real
TLS-terminating nginx proxy, the cookie came back without Secure and
Chrome silently dropped it -- breaking the SAML login flow entirely, not
just weakening it. Fixed with isSecureRequest(r), which also checks
X-Forwarded-Proto: https -- not a new trust boundary, since this handler
already assumes it sits behind exactly this kind of proxy, never
directly internet-facing.
 2026-08-15 18:06:51 -07:00
jcoffey-dev c5c68f22e9 Stop enterprise-api panicking on startup from a duplicate /healthz route 
main.go registered GET /healthz explicitly, on top of the one
queryapi.Handler.RegisterRoutes already registers -- net/http's
ServeMux panics on a duplicate pattern, so enterprise-api could never
actually start. Every previous "built" claim for this binary had only
ever been a successful go build, never a successful process start;
caught the first time this ran against a real docker-compose stack.
 2026-08-15 17:17:28 -07:00
jcoffey-dev 86afe7a005 Treat a malformed data source id as ErrNotFound, not a raw pg error 
SetDataSourceClickHouseCredentials let a non-UUID id leak Postgres's
raw 22P02 (invalid_text_representation) error past the store's
ErrNotFound boundary. A malformed id can never match a row either way,
so it should be treated the same as "no such row" rather than exposing
a database-internal error past this package's boundary. Found via a
live Postgres integration test.
 2026-08-15 17:17:22 -07:00
jcoffey-dev e15a63408a Revoke tenant users' system.* access explicitly, don't assume default-deny 
Verified live against clickhouse/clickhouse-server:24.8: a freshly
created tenant user was NOT default-denied from system.* the way the
original design assumed -- system.tables listed every tenant's
database/table names to any authenticated user regardless of grants.
Fixed with an explicit REVOKE SELECT ON system.* FROM <user> after the
existing GRANT SELECT, INSERT.

Verifying this live also surfaced a real ClickHouse behavioral split
the design didn't anticipate: system.query_log is genuinely access-
checked (the REVOKE makes it hard-deny, ACCESS_DENIED), but
system.tables is a filtered catalog view that ClickHouse 24.8 never
denies outright -- it just returns zero rows for a properly-revoked
user. Both outcomes close the actual leak. Corrected
TestProvisionedUserCannotReadSystemTables to assert what each table
actually does (hard error for query_log, verified-empty-and-no-foreign-
database-names for tables) instead of demanding a hard error from both.
 2026-08-15 17:17:11 -07:00
jcoffey-dev d9e4f22200 Fix enterprise-auth's Docker build context and ClickHouse admin access 
enterprise/Dockerfile built from enterprise/ alone, too narrow for
enterprise/go.mod's replace ../api directive once enterprise-auth
started importing api/httpserver and (transitively) api/dashboards --
confirmed broken the first time this was built with real Docker access.
Fixed to build from the repo root, matching enterprise-api/
enterprise-ingest's Dockerfiles.

Also: ClickHouse's default admin user genuinely lacked CREATE USER
privilege in docker-compose.yml -- the official image needs
CLICKHOUSE_DEFAULT_ACCESS_MANAGEMENT=1 (not the more obvious-looking
CLICKHOUSE_ACCESS_MANAGEMENT, confirmed by reading the image's own
/entrypoint.sh), which this file never set. Every tenant-provisioning
code path was correct Go that had simply never been able to
authenticate its own admin connection strongly enough to run.
 2026-08-15 17:17:03 -07:00
jcoffey-dev 2e8ab1ed6a Give chwriter.Registry periodic refresh, matching Tantivy's tracker 
Closing search's active-tenant gap last commit surfaced a real asymmetry
by comparison: chwriter.Registry's per-tenant writer map was still a
snapshot built once at enterprise-ingest startup with no refresh at all,
while search's new ActiveTenantTracker refreshes every minute. A tenant
deprovisioned after enterprise-ingest started would keep writing
successfully to ClickHouse until the next restart -- a real, disclosed
staleness gap, not matched by anything on the Tantivy side anymore.

Registry.StartRefreshing spawns a goroutine that re-lists active tenants
every minute (dataSourceRefreshInterval, same interval as search's
tracker) via a new SourceLister callback and reconciles the writer map:
opens a connection for a newly-active tenant, closes and removes one no
longer active. New connections are dialed before taking the write lock,
so a slow/unreachable ClickHouse for one newly-active tenant never
blocks WriteBatch's read lock. A refresh failure (lister error, or one
tenant's connection failing to open) logs and leaves the existing map
untouched for that tick -- the same last-known-good posture
ActiveTenantTracker already uses, so a transient rbacstore/Postgres blip
doesn't evict every other tenant's already-working writer.

WriteBatch now takes a read lock and Close takes a write lock -- the
writer map was safe unsynchronized before only because it was immutable
after New() returned; StartRefreshing makes it mutable at runtime.

enterprise-ingest/main.go extracts the existing rbacstore-row-to-
DataSource adaptation into tenantDataSourceLister, reused for both the
initial synchronous load and StartRefreshing's periodic calls, so the
two can't drift into checking different things.

Verified: the lister-error-keeps-last-known-good path is Docker-free
(same "construct a Registry directly, bypass New" trick the existing
fail-closed tests use). The actual add/remove reconciliation against
real ClickHouse connections (TestRefreshAddsNewlyActiveTenant,
TestRefreshRemovesNoLongerActiveTenant) are skip-gated live-ClickHouse
tests, same CHWRITER_TEST_CLICKHOUSE_ADDR convention as this package's
existing integration tests -- not run against a live database in this
environment.

This closes the last disclosed gap from Phase 4's write-routing work:
both storage engines now share the same one-minute active-tenant
staleness bound instead of one being materially staler than the other.
 2026-08-14 23:55:40 -07:00
jcoffey-dev 088677643f Close search's active-tenant write-routing gap with a polled allowlist 
search/src/consumer.rs's write-routing (built last pass) had no active-
tenant check at all: IndexRegistry.resolve() would open-or-create an
index directory for any syntactically-valid tenant_id, active or not --
unlike ClickHouse's chwriter.Registry (an active-tenants-only snapshot
built at enterprise-ingest startup) or the read side (gated by
searchclient.TenantChecker, a direct rbacstore query). search is AGPL
core with no Postgres access and no enterprise/ import allowed, so it
needed a network boundary instead -- the same shape ingest's
TenantResolver already uses against enterprise-auth, just Rust calling
Go instead of Go calling Go.

New GET /internal/active-tenants endpoint on enterprise-auth
(rbacstore.ListActiveTenantIDs + authhandler.handleActiveTenants),
gated on a RoleService Bearer credential -- server-to-server auth, the
same shape alerting presents to api, minted via the already-generic
enterprise-auth -mint-service-token search. search/src/tenants.rs's
ActiveTenantTracker polls it every 60s, blocking startup on the first
fetch succeeding (fail-closed cold start -- a control-plane outage at
boot must not silently accept every tenant_id) and keeping the last-
known-good set on any later refresh failure (a transient blip shouldn't
stop every tenant's indexing, only prevent the allowlist from growing/
shrinking until connectivity resumes). consumer.rs refuses any tagged
record whose tenant isn't in the polled set, before ever calling
resolve() -- IndexRegistry itself stays policy-free, matching the same
mechanism/policy split clickhousewriter.Writer vs. chwriter.Registry
already draws on the ClickHouse side.

Off unless ENTERPRISE_AUTH_URL/ENTERPRISE_AUTH_SERVICE_TOKEN are both
set (search/src/config.rs rejects exactly one being set) -- every
existing deployment is unaffected.

Verified with real HTTP round trips in this environment: tenants.rs's
tests exercise real reqwest requests (actual Authorization: Bearer
header, actual JSON parsing) against a hand-rolled dependency-free TCP
test server, including both fail-closed paths (rejected first fetch,
unreachable server). authhandler's new tests cover the credential-kind
distinction this endpoint exists to enforce -- a real human session,
even for a genuine Owner, must not satisfy a check meant for a service
identity.

One asymmetry remains, disclosed rather than fixed: chwriter.Registry's
snapshot still never refreshes (stale until enterprise-ingest restarts),
while ActiveTenantTracker's 60s poll gives Tantivy a materially tighter
staleness window. Neither is a live per-write check -- that would mean
a database/HTTP round trip per record, a throughput cost neither
implementation accepts -- so both have some staleness window by design;
the gap between the two windows is what's disclosed, not a claim either
is fully live.
 2026-08-14 23:47:25 -07:00
jcoffey-dev 5a845f06ee Add rbacstore.TransferOwner and enterprise-auth -transfer-owner-* 
RevokeMembership refuses to revoke a tenant's current Owner (would leave
tenants.owner_user_id dangling), but there was no way to actually hand
ownership to someone else -- only the raw SetOwner primitive, which
-grant-membership-role=owner used without downgrading whoever held
Owner before, leaving tenant_memberships claiming two owners while
owner_user_id can only name one. Named as a real, disclosed gap in
docs/phase-4-runbook.md's "Known gaps".

rbacstore.TransferOwner closes it: downgrades the current owner's
membership to admin, promotes the new owner, and updates
tenants.owner_user_id, all in one pgx transaction -- this package's
first use of one. Every other mutation here is a single independent
statement because nothing else needs more than one row to agree; this
does, for the same reason RevokeMembership's doc comment already gives
for refusing to revoke Owner in the first place.

-grant-membership-role=owner now refuses when a *different* owner
already exists, pointing at the new -transfer-owner-tenant/
-transfer-owner-user-email flags instead of silently producing the
inconsistent state -- it remains correct, unchanged, for a tenant's
first owner assignment.

Verified with the same skip-gated live-Postgres discipline as the rest
of this package (TestTransferOwnerMovesOwnershipAndDowngradesPrevious
Owner proves the downgrade is real by then successfully revoking the
former owner's now-non-Owner membership; two refusal-path tests cover
no-current-owner and transfer-to-self) -- not run against a live
database in this environment, same disclosed gap as everything else
here.
 2026-08-14 23:30:05 -07:00
jcoffey-dev abeee0076b Build and browser-verify the tenant-picker frontend page 
web/src/routes/select-tenant now calls enterprise-auth's existing
GET /auth/memberships / POST /auth/select-tenant protocol (built earlier
this phase, previously called only from Go tests) via
fetch(..., {credentials: 'include'}) -- new listMemberships/selectTenant
functions in $lib/api.ts, using a dedicated request helper that reads
plain-text error bodies (loginhandler's http.Error responses), unlike
every other request helper in that file which expects JSON.

Credentialed cross-origin fetch needed CORS enterprise-auth didn't have:
api/httpserver.WithCORS's wildcard-friendly default can't be combined
with a credentialed request at all (browsers refuse to honor
Access-Control-Allow-Origin: "*" on one) -- added WithCredentialedCORS
(literal origin, Access-Control-Allow-Credentials: true) alongside it,
wired into enterprise-auth via a new CORS_ALLOWED_ORIGIN config var
defaulting to POST_LOGIN_REDIRECT_URL (web's own origin, the same
default pattern SELECT_TENANT_REDIRECT_URL already used).

adapter-static's route crawler doesn't discover a page nothing links to
(this one is only ever reached via enterprise-auth's redirect) -- fixed
with select-tenant/+page.ts's `export const prerender = true`, the same
declaration every other route already has.

Genuinely verified in a real browser in this environment, not just
type-checked: a throwaway Node server standing in for enterprise-auth's
exact wire contract (including its plain-text error bodies), driven
through the full flow via mcp__claude-in-chrome -- cross-origin
pending-login cookie set, credentialed preflight + GET/POST round trip,
a real click choosing a tenant, the post-selection redirect, and the
missing/expired-cookie error path rendering the backend's actual
message. No Docker or live Postgres/IdP needed, since the point was
exercising web's own fetch/CORS/cookie wiring, not enterprise-auth's
internals (already covered by loginhandler's own tests).

This closes the tenant-picker as the last named gap in Phase 4. What's
left is the already-disclosed live-verification caveat shared by every
Postgres/ClickHouse-backed piece and both SSO protocols: none of this
has run against a real database, external IdP, or multi-container
deployment in this environment.
 2026-08-14 23:02:09 -07:00
jcoffey-dev bdd42e06f6 Build per-tenant Tantivy write-routing, closing the last ingest write gap 
search/src/consumer.rs now resolves each record's tenant_id Kafka header
through the same IndexRegistry the read side (search/src/registry.rs +
enterprise/internal/searchclient) already used, and writes into that
tenant's own index instead of always the default one. The periodic
Tantivy commit now commits every tenant index that's actually seen a
write (IndexRegistry::commit_all), not just the default index.

Unlike ClickHouse, this needed no "second binary": Tantivy has no grant
system to gate a commercially-licensed credential behind, so
IndexRegistry already lived directly in this AGPL-core search binary --
there was never an import-boundary reason to split the write side into
an enterprise/ binary the way chwriter/enterprise-ingest was for
ClickHouse. Read and write simply share one registry.

Because Tantivy is an embedded library, this is genuinely verified in
this environment, not just written: registry.rs's
commit_all_commits_default_and_every_opened_tenant_index writes into the
default index plus two tenant indices, confirms nothing is searchable
pre-commit, then confirms all three are post-commit. consumer.rs's
tenant_id_from_headers is factored out as a small pure helper (mirroring
ingest/consumer.tenantIDFromHeaders) with its own unit tests, plus a
guard test against the "tenant_id" header-key literal drifting from the
Go side's -- the same guard-test pattern ingest/cmd/ingest already used
for its own two Go copies of the constant, now mirrored a third time
across the language boundary.

One gap is disclosed, not fixed, by this change: unlike chwriter.Registry
(an active-tenants-only snapshot built at enterprise-ingest startup, so
an unrecognized tenant_id is refused outright) and unlike the read side
(gated by searchclient.TenantChecker), this consumer's registry.resolve()
call has no active-tenant check at all -- search has no Postgres access
to check tenant status against. A still-valid-but-should-be-revoked
ingest credential can cause an index directory to be created for a
tenant that's no longer active. Narrow blast radius (an orphan, isolated,
empty index, not cross-tenant leakage, and only reachable with a real
signed credential), but real -- see registry.rs's doc comment on
resolve(). Closing it fully would mean giving search some way to learn
which tenants are active without an enterprise/ import, which isn't
designed yet.

This closes the last of Phase 4's ingest write-routing gaps (ClickHouse
was closed last commit). The one remaining gap in the whole phase is now
the tenant-picker frontend page, deliberately deferred earlier in this
phase as out of scope for this environment.
 2026-08-14 20:16:09 -07:00
jcoffey-dev 1de77b969f Build per-tenant ClickHouse write-routing for ingest (Tantivy still deferred) 
ingest tags every record with a tenant_id Kafka header (built previously),
but nothing consumed it to actually route the write. This closes that for
ClickHouse: enterprise/cmd/enterprise-ingest (a second binary, mirroring
enterprise-api) reuses ingest/consumer's own flush loop unchanged, with
enterprise/internal/chwriter.Registry -- a per-tenant clickhousewriter.Writer
registry -- swapped in as the writer. A batch pulled from the single shared
Redpanda topic can mix records from many tenants, so WriteBatch groups by
TenantID and dispatches each group to its own tenant's connection, fail-
closed on an empty or unrecognized tenant_id.

ingest/consumer and ingest/clickhousewriter move out of internal/ (same
reason api/internal/* moved earlier this phase: enterprise/ can't import
anything under another module's internal/). Their New() constructors now
take small local Config structs instead of ingest/internal/config types,
so enterprise/ doesn't need that import either.

Building this surfaced a real bug: tenantprovision.ProvisionClickHouse
only granted SELECT on a tenant's ClickHouse user, correct for chrunner's
read-only use but not enough for chwriter reusing the same credential to
write -- every real per-tenant write would have failed closed with a
permission error. Fixed by widening the grant to SELECT, INSERT; no
cross-tenant boundary is crossed by also allowing INSERT within a
tenant's own database.

Helm gates enterprise-ingest's Deployment on the same
ingest.requireTenantCredential flag that already gates tag validation --
write-routing is meaningless without tagging already being required, so
they're one decision, not two. docker-compose.yml's version is a
disclosed, weaker approximation: it can't achieve Helm's genuine
-mode=server/-mode=consumer split, so with the enterprise profile active
both ingest and enterprise-ingest independently consume every message
via different consumer groups -- harmless duplication for local
verification only.

Not built: Tantivy's independent Redpanda consumer (search/src/consumer.rs)
still doesn't read the tenant_id header at all -- every record still lands
in the one shared index regardless of tenant. Not run: the live-ClickHouse-
gated tests (chwriter's cross-tenant routing test, tenantprovision's INSERT
regression test) -- no Docker/database access in this environment; they're
correct Go that has never executed, disclosed as such in docs/security/
threat-model.md and docs/phase-4-runbook.md §14.
 2026-08-14 19:26:09 -07:00
jcoffey-dev 17fdc212c2 Give ingest a real tenant identity (write-routing deferred, disclosed) 
Ingest tenant-awareness was named "undesigned, not just unbuilt" across
CLAUDE.md/threat-model.md/the runbook since early Phase 4 -- the last
major standing gap. Scoping was agreed via AskUserQuestion: a
config-supplied tenant_id + shared-secret token ingest validates
(smaller real implementation, no new PKI), over per-tenant mTLS
certs. This change builds that identity mechanism end to end and
attaches it to every record at the point it enters the system; it
deliberately does NOT build per-tenant write-routing for ClickHouse or
Tantivy -- that's real, separately-scoped follow-up work, disclosed
explicitly everywhere this was previously called undesigned, not
silently left half-done.

New pieces:

- metadata/migrations/0034 + enterprise/internal/rbacstore/
  ingest_credentials.go: a per-tenant bearer credential, only its
  SHA-256 hash ever persisted (same reasoning a password gets hashed,
  not stored raw) -- CreateIngestCredential returns the plaintext
  exactly once, ValidateIngestCredential/RevokeIngestCredential/
  ListIngestCredentialsForTenant round it out.
- enterprise-auth gains -create-ingest-credential-tenant/
  -list-ingest-credentials-tenant/-revoke-ingest-credential (same
  offline-operator-flag shape as every other credential-minting flag in
  this binary) and a new POST /internal/authorize-ingest endpoint
  (internal/authhandler) validating a presented token and resolving its
  tenant -- a genuinely different credential type from session-backed
  /internal/authorize, so it doesn't touch session.Manager at all.
- ingest (AGPL core) gains an optional TenantResolver
  (internal/grpcserver, nil by default) and its HTTP client
  implementation (internal/tenantresolver.HTTPResolver) -- a plain HTTP
  call to enterprise-auth's new endpoint, never an enterprise/ import,
  same "network boundary, not import boundary" shape
  api/authz.HTTPAuthorizer already uses for the query path.
  PushBatch now requires an `authorization: Bearer <token>` gRPC
  metadata entry once a resolver is configured, fails the whole batch
  closed on a missing/invalid credential (never falls back to "no
  tenant"), and attaches the resolved tenant ID to every record as a
  `tenant_id` Kafka message header before producing it.

Verified with real round trips at every layer, no Docker needed:
rbacstore's credential CRUD (skip-gated on live Postgres, same as every
other rbacstore integration test this phase), authhandler's new
endpoint (real HTTP via httptest, including the regression test that a
session token must not validate as an ingest credential), tenantresolver
(real HTTP client against httptest, same pattern as
authz.HTTPAuthorizer's own tests), and grpcserver's PushBatch (fake
resolver/producer -- no resolver leaves messages unchanged, a configured
resolver attaches the right header or fails closed on a bad/missing
token).

Helm: ingest.requireTenantCredential (default false) is a deliberate,
separate opt-in from enterprise.enabled -- turning ENTERPRISE_AUTH_URL
on for ingest requires every agent to already hold a credential or be
refused outright, so it must not default on just because
enterprise.enabled does (same reasoning api.yaml's ENTERPRISE_AUTH_URL
isn't tied to enterprise.enabled directly either). docker-compose.yml
leaves it unset, same as ever.

Docs updated everywhere this was called "undesigned": CLAUDE.md,
docs/architecture.md, docs/security/threat-model.md (including its
summary table, now split into "identity: built" vs "write-routing: not
yet"), docs/phase-4-runbook.md (new §13), enterprise/README.md.
 2026-08-14 15:21:55 -07:00
jcoffey-dev d2c76aa3a4 Build the tenant-picker backend protocol (no frontend yet, by design) 
A multi-membership identity (belongs to more than one tenant) used to
get a flat 501 refusal -- named as undesigned future work across
CLAUDE.md/threat-model.md/the runbook since early Phase 4. Scope for
this change was agreed via AskUserQuestion: backend protocol only,
fully verified via real HTTP round trips, not the actual picker page --
web has zero session/cookie-handling code today (confirmed while
researching this), so building that is separately-scoped, unverifiable
frontend work in this environment (no live backend, no browser).

session.Manager gains IssuePendingLogin/ValidatePendingLogin, a second
JWT token type proving identity without committing to a tenant yet
(10-minute TTL). PendingLoginClaims is deliberately a distinct Go type
from Claims, and -- caught by this change's own test suite before it
shipped -- needed a JSON field name disjoint from Claims.UserID's
"user_id" too: go-jose's unmarshal is happy to populate a struct from
any token whose claims happen to share a key, so a real session token
would otherwise have parsed successfully as a pending login. Fixed via
"pending_user_id" instead; both directions (session-as-pending,
pending-as-session) now have regression tests.

rbacstore.ListMembershipsWithTenantForUser joins tenant_memberships
with tenants, since a picker needs display names, not just IDs.

loginhandler.resolveIdentity's multiple-membership branch no longer
errors -- finishLogin routes it into startTenantSelection instead,
which issues a pending-login cookie (Path=/auth, so it's never sent on
ordinary requests) and redirects to a new configurable
SelectTenantRedirectURL (defaults to {POST_LOGIN_REDIRECT_URL}/select-
tenant). Two new routes complete the round trip: GET /auth/memberships
lists the pending identity's real tenant options, and POST
/auth/select-tenant re-derives the role for the chosen tenant
server-side (never trusts a client-supplied role, refuses a tenant_id
outside the identity's actual memberships with 403) before issuing the
real session -- responding with JSON {"redirect_url": ...}, not a
redirect, since a POST/fetch caller should control its own navigation.

Verified with the same real-fake-IdP tests the rest of this package
uses (coreos/go-oidc's oidctest, crewjam/saml's samlidp): the full
login -> pending cookie -> GET /auth/memberships -> POST
/auth/select-tenant -> real session round trip for both protocols, plus
negative paths (missing/expired pending cookie, a tenant_id outside
membership, a real session token rejected as a pending login and vice
versa). ErrMultipleMemberships is removed -- it's not an error path
anymore.

Docs updated in lockstep: CLAUDE.md, threat-model.md (including its
summary table), phase-4-runbook.md (new §12), enterprise/README.md
(new "Tenant selection" section, explicit about what's still not built
and why: no session handling in web, no CORS on enterprise-auth).
 2026-08-14 14:04:37 -07:00
jcoffey-dev cfcbc77507 Add enterprise-auth -revoke-membership-*/-list-memberships-tenant flags 
Rounds out the tenant_memberships operator flags added earlier this
phase (-create-tenant/-grant-membership-*): grant had no way to undo
itself, and there was no way to see who was actually in a tenant
without querying Postgres directly.

rbacstore.RevokeMembership deletes a tenant_memberships row, but
refuses to revoke a tenant's current Owner -- Owner is also a
dedicated tenants.owner_user_id column (SetOwner), so deleting that
membership without transferring ownership first would leave
owner_user_id pointing at a user with no membership in the tenant at
all. Ownership transfer is a deliberate, separate action per the RBAC
matrix ("Transfer tenant Owner -- Owner only"), not a side effect of
revoking access.

rbacstore.ListMembershipsForTenant is ListMembershipsForUser's
inverse -- joined with users so the result is actually useful (email,
display name), not just a bare user ID list.

enterprise-auth gains -revoke-membership-tenant/
-revoke-membership-user-email (both required together, same shape as
-grant-membership-*) and -list-memberships-tenant (prints tab-separated
id/email/display-name/role to stdout and exits -- an operator
convenience, not a machine-readable API; this binary deliberately has
no admin HTTP surface, per its own doc comment on why). Changing a
role is unchanged: re-run -grant-membership-* with a different
-grant-membership-role, SetMembership's upsert already handles it.

Covered by five new skip-gated integration tests in rbacstore_test.go
(RBACSTORE_TEST_POSTGRES_ADDR), including the Owner-revocation refusal
and cross-tenant leak check for ListMembershipsForTenant -- not run
against a live database in this environment, same disclosed gap as the
rest of this phase's Postgres-backed work. Docs updated:
phase-4-runbook.md's known-gaps bullet, enterprise/README.md's
bootstrap walkthrough and package layout table.
 2026-08-14 09:11:01 -07:00
jcoffey-dev 823f5d48d1 Unify the Tenant CRD with enterprise-api -provision-tenant (lightweight) 
Closes a gap named across CLAUDE.md/docs/architecture.md/deploy/README.md
since early Phase 4: the operator's Tenant CRD and -provision-tenant
were two disconnected mechanisms. The operator's reconciler generated a
K8s Secret with a locally-generated random password that authenticated
against nothing (nothing ever called ClickHouse to create a matching
user), and unconditionally claimed status.phase=Active the moment a
Tenant object existed -- actively misleading, not just incomplete.

Two unification shapes were considered (surfaced to the user via
AskUserQuestion, given the real difference in blast radius): the
operator's reconcile loop becoming a second real actor (new Postgres +
ClickHouse admin credentials flowing into the K8s controller, plus real
reconcile-loop idempotency/retry design for an inherently one-shot
external side effect), or keeping -provision-tenant as the sole real
actor and having it also sync its result into the CRD. Went with the
lighter option.

enterprise/internal/tenantcrd (new): a Syncer using the K8s dynamic
client (unstructured.Unstructured + a GroupVersionResource, not
deploy/operator's typed Tenant struct -- avoids a cross-module Go
dependency between two independently-versioned modules for one type).
Upserts the Tenant object, creates/updates a Secret with the *real*
ClickHouse credentials owned by that Tenant via an OwnerReference, then
patches status.{clickHouseDatabaseName,clickHouseSecretRef,
tantivyIndexPath}. Idempotent and safe to retry: never rotates a
credential across a re-sync, never overwrites a pre-existing
spec.displayName a human/GitOps process set.

cmd/enterprise-api/main.go's runProvisionTenant calls Sync when
TENANT_CRD_NAMESPACE is set (empty = no-op, same shape as every other
optional dependency in this codebase). Its "already active" refusal is
now split: ClickHouse re-provisioning is still refused (rotating a live
credential would break every open connection for no benefit), but CR
sync alone is now retryable using the credentials already on file in
rbacstore -- needed for retrying a previously-failed sync, or
backfilling CR sync for a tenant provisioned before this existed.

deploy/operator's reconciler rewritten to match: it never claims
PhaseActive on its own initiative anymore, only once
status.ClickHouseDatabaseName is non-empty (the field -provision-tenant,
and only -provision-tenant, sets). Phase is now a pure function of
{spec.suspended, status.ClickHouseDatabaseName != ""} recomputed every
reconcile, not toggled in place -- fixes a related bug the old code
would have hit once suspension was involved: un-suspending an
already-provisioned tenant needs to return straight to Active, which
isn't derivable from "last observed phase was Suspended" alone. The
reconciler no longer creates or manages any Secret, dropped its
`secrets` RBAC grant entirely, and gained zero new dependencies.

Helm chart: enterprise-api gets its own ServiceAccount/Role/RoleBinding
(get/list/create tenants, get/update/patch tenants/status, get/create/
update secrets -- least-privilege, scoped to the release namespace, not
a ClusterRole) and a TENANT_CRD_NAMESPACE env var, both gated on
tenantOperator.enabled. tenant-operator's ClusterRole loses the
secrets grant it no longer needs.

Verified in this environment: enterprise/internal/tenantcrd's tests run
against k8s.io/client-go's fake dynamic + typed clientsets (real client
library, fake transport, no cluster needed); deploy/operator's rewritten
tenant_controller_test.go runs against controller-runtime's fake
client, including new regression tests for the "must not claim Active
without confirmation" and "un-suspending returns to Active, not
Provisioning" properties; helm template + parsing the rendered YAML
confirms the RBAC split renders exactly as designed under both
tenantOperator.enabled=true/false. Not verified: an actual
-provision-tenant run against a real cluster with the operator watching
(no live cluster in this environment, same disclosed limitation as the
rest of /deploy). Docs updated in lockstep: CLAUDE.md, docs/architecture.md,
deploy/README.md, deploy/helm/sentry/README.md (including a corrected
"Trying the two-tenant example" walkthrough), phase-4-runbook.md (new
§11), enterprise/README.md. Also fixed two unrelated stale claims found
along the way: docs/architecture.md still said docker-compose.yml ran
plain api unconditionally (fixed in an earlier commit, doc not updated
then), and enterprise-api's own main.go doc comment still said Helm/
docker-compose wiring wasn't built yet.
 2026-08-14 09:07:10 -07:00
jcoffey-dev 8d7326fc6a Make docker-compose.yml enforce the api/enterprise-api binary swap 
Closes the local/dev half of a gap named repeatedly across this
phase's docs: Helm already made api/enterprise-api mutually exclusive
(same enterprise.enabled flag that turns on RBAC/audit/SSO, both
rendering to the same Service name/port); docker-compose.yml let both
run side by side, with nothing actually pointing at enterprise-api by
default.

Mechanism: both services now carry a `profiles` entry (single-tenant /
enterprise), selected via COMPOSE_PROFILES -- a new checked-in .env
sets single-tenant as the zero-config default (unchanged behavior for
anyone who doesn't touch it), and `COMPOSE_PROFILES=enterprise docker
compose up` swaps to enterprise-api instead. Docker Compose profiles
are purely additive (no "profile X excludes service Y" primitive), so
true exclusivity comes from both being profile-gated with no shared
default profile, not from one excluding the other directly.

Mirrors Helm's same-Service-name trick so alerting's API_QUERY_URL and
web's VITE_API_BASE_URL need zero conditional logic either way:
enterprise-api now maps host port 8080 (was 8083, its own binary
default -- overridden via HTTP_LISTEN_ADDR) and carries a
`networks.default.aliases: [api]` entry, so whichever binary is
actually running answers on the same compose-network hostname and host
port. alerting's and web's depends_on for api/enterprise-api are now
`required: false` (Compose's supported "optional dependency" shape) --
without it, compose errors on the inactive one rather than just
skipping it, since depends_on doesn't otherwise know about profiles.

Verified for real in this environment via `docker compose config`
(renders and validates the merged YAML without needing a daemon):
confirmed api/enterprise-api never both appear in --services output
for either profile selection, confirmed enterprise-api's rendered
block has port 8080/alias "api"/HTTP_LISTEN_ADDR ":8080" when the
enterprise profile is active, and confirmed `docker compose run
enterprise-api ...`/`docker compose build enterprise-api` (used by
enterprise/README.md's and phase-4-runbook.md's provisioning steps)
still work regardless of the active profile -- explicit service
references bypass profile filtering, confirmed by the commands
reaching a daemon-connection permission error rather than a
profile-resolution error. Not verified: an actual `docker compose up`
against a real daemon, still unavailable in this environment.

Docs updated in lockstep -- CLAUDE.md, threat-model.md (including its
summary table), phase-4-runbook.md (new §10a, §8's provisioning
commands updated for the new port/profile), enterprise/README.md.
 2026-08-14 08:17:41 -07:00
jcoffey-dev 2e698f5623 Close the last tenant-isolation adversarial probe (mid-provisioning tenants) 
Phase 4 task 8's verification plan named four adversarial probes;
three were closed earlier this phase, the fourth (an evaluator tick,
or any other caller, hitting a tenant that exists but hasn't reached
the active+credentialed gate yet -- must be refused, not served) was
still an explicitly-skipped stub in
api/queryapi/tenant_isolation_gap_test.go.

Investigating it found the two storage engines needed genuinely
different treatment:

- ClickHouse (enterprise/internal/chrunner) already had this property
  structurally, for free: Registry is built once at startup from
  rbacstore.ListProvisionedDataSources, which already filters to
  active+credentialed tenants only, so a mid-provisioning tenant is
  simply absent from the connection map. New test
  TestRegistryRefusesMidProvisioningTenant proves this without Docker
  -- an empty DataSource list never dials ClickHouse, so this genuinely
  runs in this environment, unlike every other test in that file.

- Tantivy (search/src/registry.rs's IndexRegistry) was a real, different
  gap, not just an unverified assumption: it opens-or-creates an index
  for any syntactically-valid tenant_id on first request, because it's
  a separate process with no Postgres access and structurally can't
  know which tenants are actually provisioned. A query against a
  mid-provisioning tenant would have silently succeeded with zero
  results from a freshly-created empty index -- "ambient success"
  indistinguishable from "no matching logs," exactly the failure mode
  this item was worried about.

Fixed the Tantivy gap with a new enterprise/internal/searchclient.
TenantChecker interface (backed by a new rbacstore.TenantIsActive,
implemented structurally, no new import edge needed), consulted before
every gRPC call: Client.Search now refuses a non-active tenant before
it ever reaches `search`. Dial's signature gained a required
TenantChecker parameter; enterprise-api's main.go passes its existing
rbacstore.Store (already satisfies the interface). Verified Docker-free
via searchclient's existing real-in-process-gRPC-server test harness
(TestSearchRefusesMidProvisioningTenant, plus
TestSearchPropagatesTenantCheckerError for the fail-closed-on-error
case) -- both genuinely run in this environment, same bar as the rest
of the Tantivy isolation work.

rbacstore.TenantIsActive itself has two new skip-gated live-Postgres
tests (TestTenantIsActive, TestTenantIsActiveNonexistentTenant) --
disclosed as not run against a live database here, same gap as the
rest of this phase's Postgres-backed pieces.

api/queryapi/tenant_isolation_gap_test.go rewritten from a checklist
with one skipped stub to a full accounting of all four now-closed
probes. Docs updated in lockstep: CLAUDE.md, threat-model.md,
phase-4-isolation-design.md (implementation note added after its
original sign-off), phase-4-runbook.md (§9), enterprise/README.md.
 2026-08-14 08:07:52 -07:00
jcoffey-dev b8b6a8fd7b Add enterprise-auth -create-tenant/-grant-membership-* operator flags 
Replaces the manual psql INSERT dance phase-4-runbook.md's §3a/§3b
documented for bootstrapping the very first tenant_memberships row
(create the tenant, log in once so UpsertUserBySSO creates a users row,
hand-write an INSERT with that user's UUID). Two new offline operator
flags, same "gated by access to enterprise-auth's own environment, not
a network-reachable endpoint" shape as -mint-service-token and
enterprise-api's -provision-tenant:

- -create-tenant=<id> [-display-name=<name>]: creates a tenant row in
  rbacstore (control-plane only -- pair with enterprise-api
  -provision-tenant separately for ClickHouse/Tantivy data-plane
  provisioning, still two operator actions today, a named gap this
  doesn't unify). Refuses to run twice for the same id.
- -grant-membership-tenant/-grant-membership-user-email/
  -grant-membership-role: grants a tenant_memberships row by email
  instead of requiring the operator to hand-look-up a UUID. The user
  must already exist (attempted an SSO login at least once -- this flag
  deliberately never creates a user itself, since that identity has to
  come from a real IdP round trip). role=owner also calls SetOwner,
  since Owner is a dedicated tenants.owner_user_id column, not just the
  highest tenant_memberships role.

Deliberately kept as offline flags rather than an authenticated HTTP
admin API: an HTTP endpoint would have to solve "who's allowed to
create the very first tenant/membership" itself, a real bootstrap
problem the offline-flag pattern already used elsewhere in this binary
sidesteps entirely.

New rbacstore.GetUserByEmail supports the email-based lookup (email is
already the natural key UpsertUserBySSO upserts on). Covered by two new
skip-gated integration tests in rbacstore_test.go, same
RBACSTORE_TEST_POSTGRES_ADDR convention as the rest of this package --
not run against a live database in this environment, consistent with
everything else in this phase's Postgres-backed work. No dedicated test
for the main.go flag handlers themselves, matching the existing
precedent for -mint-service-token/-provision-tenant (neither has one
either).

Docs updated: phase-4-runbook.md's §3a/§3b bootstrap steps and its
"known gaps" list, enterprise/README.md gets a new "Bootstrapping a
tenant and its first human user" section and a stale "there's no login
flow to issue a human session yet" line (obsolete since OIDC/SAML login
shipped) is fixed.
 2026-08-14 07:57:33 -07:00
jcoffey-dev 243f4dc2ab Enforce per-resource dashboard grants (RBAC matrix's own/granted qualifier) 
api/dashboards' handler previously enforced only tenant-baseline role
(RoleEditor+), so any Editor could edit/delete any dashboard in their
tenant -- the matrix's "(own/granted)" qualifier was explicitly named
as unbuilt in this handler's own doc comment. This closes that gap.

New core interface api/dashboards.PermissionStore (nil-safe, same "not
wired == no-op" shape as authz.Authorizer) resolves a per-resource
dashboard_permissions grant. canEditDashboard now requires the
identity be Admin/Owner, the dashboard's creator, or hold a grant of at
least Editor; canManageGrants is deliberately stricter (creator or
Admin/Owner only, never grant-derived access) so a user who can edit a
dashboard only because of a grant can't extend or re-grant that access
to themselves or others. Wired handlers: PUT/DELETE
/dashboards/{id}/permissions/{userId}, GET .../permissions.

Two real bugs found and fixed while wiring this up, before any of it
touched a live database:
- handleCreate/handleImport never stamped created_by from the
  authenticated identity, so every dashboard was owned by "anonymous"
  regardless of who made it -- the ownership check would have been
  meaningless. Also fixed: ImportDashboard trusted the exported JSON's
  created_by verbatim, so re-importing someone else's export would
  leave the actual importer unable to edit their own copy.
- metadata/migrations/0024_create_dashboard_permissions.sql's CHECK
  constraint diverged from /docs/phase-4-rbac-design.md's schema
  (allowed role='admin', nullable granted_by). Reconciled via
  0033_restrict_dashboard_permissions_role.sql: Admin/Owner already
  have tenant-wide access so a resource-level "admin" grant is
  meaningless, and every real grant now always has an attributable
  granter.

enterprise/internal/rbacstore gets the storage side: raw CRUD
(dashboard_permissions.go) plus DashboardPermissions
(dashboards_adapter.go), an adapter implementing
api/dashboards.PermissionStore -- same pattern as audit.QueryAPILogger
over queryapi.AuditLogger. Wired into enterprise/cmd/enterprise-api
only; plain api/cmd/api passes nil (ownership/Admin checks still work
via the nil-permissions fallback, just without the "granted" bonus).

Verified: the full own/granted/admin/creator matrix, including the
granted-editor-cannot-manage-grants regression, passes against a fake
PermissionStore (api/dashboards/handler_test.go, all existing tests
also still pass unmodified in behavior). Real integration tests exist
in enterprise/internal/rbacstore/rbacstore_test.go (skip-gated on
RBACSTORE_TEST_POSTGRES_ADDR, same convention as every other
Postgres-backed piece this phase) but have not run against a live
database in this environment -- disclosed in threat-model.md,
phase-4-runbook.md, and enterprise/README.md alongside every other
piece carrying the same gap. Also fixed a stale path in
phase-4-runbook.md's dashboards-tenant-scoping section
(./internal/dashboards/... -> ./dashboards/..., stale since that
package moved out of api/internal/ earlier in this phase).
 2026-08-14 07:11:18 -07:00
jcoffey-dev 08a90a27aa Build SAML login (enterprise/internal/loginhandler), mirroring OIDC 
Adds GET /auth/saml/login + POST /auth/saml/acs alongside the existing
OIDC pair, both converging on the same upsert-user/resolve-tenant/
issue-session path. loginhandler.New now takes an optional
*saml.ServiceProvider, RegisterRoutes registers each protocol's routes
independently so either, both, or neither can be configured. SAML's
replay/unsolicited-response defense (InResponseTo, standing in for
OIDC's state) is carried via a SameSite=None sentry_saml_request cookie
-- None because the ACS endpoint receives a cross-site POST from the
IdP's origin, which SameSite=Lax cookies are never sent on.
enterprise-auth's main.go now fetches+parses SAML_IDP_METADATA_URL at
startup (samlsp.FetchMetadata) and wires the result through.

Verified to the same bar as OIDC: a real fake IdP
(crewjam/saml/samlidp, genuine XML signing/verification) drives the
full login->ACS->session-cookie round trip and negative paths (bad
InResponseTo, missing request cookie, missing email/NameID, no/multiple
tenant memberships), all in loginhandler/saml_test.go, no Docker
needed. The login-form HTML is bypassed by pre-seeding a saml.Session
directly into samlidp's session store and presenting the matching
`session` cookie -- an IdP-supported shortcut (confirmed by reading
GetSession), the same "skip the UI, keep the crypto real" approach
oidctest gave the OIDC tests.

Writing that test caught two real bugs in internal/saml.ParseResponse,
both fixed here: it never called r.ParseForm() before reading the
POSTed SAMLResponse field, so every real ACS POST would have silently
decoded an empty response; and its email-attribute matching missed
urn:oid:0.9.2342.19200300.100.1.3 (the standard LDAP "mail" OID), which
is what an IdP sends by default absent an explicit
AttributeConsumingService request for "email" -- exactly what
samlidp's own DefaultAssertionMaker does, and plausibly what real IdPs'
default SAML app templates do too.

Docs (CLAUDE.md, threat-model.md, architecture.md, enterprise/README.md,
phase-4-runbook.md, docker-compose.yml's enterprise-auth comment)
updated in lockstep: SAML login moves from "protocol mechanics only" to
"built, verified with a real fake IdP, not yet tried against a real
external IdP or a running enterprise-auth container" -- the same
disclosed gap OIDC already carried.
 2026-08-14 06:39:36 -07:00
jcoffey-dev ba2276aa1a Phase 4: real Tantivy per-tenant isolation (search/src/registry.rs, enterprise/internal/searchclient) 
Closes the last named "isolation mechanism" gap: search.proto gains a
tenant_id field on SearchRequest; search/src/registry.rs's IndexRegistry
resolves it to an on-demand-opened, per-tenant Tantivy index (empty
tenant_id keeps today's single default index, so this is purely
additive); enterprise/internal/searchclient sets that field from the
authenticated request identity in ctx, mirroring chrunner's exact
fail-closed "never a parameter" shape. Wired into enterprise-api in
place of the shared api/searchclient.

Unlike the ClickHouse pieces from the previous two commits, this one is
genuinely verified end to end in this environment: Tantivy is an
embedded library, not a networked service, so both the Rust index
registry (cargo test, cargo clippy --all-targets -- -D warnings, both
clean) and the Go client (a real in-process gRPC server) could actually
run. registry.rs's tenant_index_is_isolated_from_default_and_other_tenants
seeds three real indices with the same term and confirms a tenant-scoped
search returns only that tenant's document -- item 3 of the isolation
design doc's verification plan, closed for real, not just written.

With both ClickHouse and Tantivy isolation now built, the single largest
remaining gap is no longer a missing mechanism: it's that nothing forces
or flags whether a deployment actually runs enterprise-api instead of
plain api, and that ingest itself has no tenant concept for either
storage engine (every record still lands in the one shared database/
index no matter what -- undesigned, not just unbuilt). Updated the
threat model, architecture doc, CLAUDE.md, and both READMEs accordingly.
 2026-08-13 23:16:22 -07:00
jcoffey-dev 1fab02abd5 Phase 4: real OIDC human login (enterprise/internal/loginhandler) 
Closes the other major named gap from this phase: until now, there was
no way for a human to actually log in -- only /alerting's RoleService
credential could be minted. GET /auth/oidc/login and GET
/auth/oidc/callback drive the real coreos/go-oidc flow already wired in
enterprise/internal/oidc: CSRF state in a short-lived cookie, code
exchange, ID token verification, upserting a users row, resolving
tenant/role from exactly one tenant_memberships row (refusing outright
on zero or more than one, rather than guessing), and issuing a real
session cookie.

Unlike everything else built this phase, this one is genuinely verified
end to end: the tests spin up coreos/go-oidc's own oidctest fake IdP,
which signs real RS256 ID tokens, and drive the full login->callback->
session-cookie round trip through actual signature verification -- no
live database or Docker needed, so nothing here is asserted without
having actually been run in this session. Also fixes a real bug caught
while wiring this into enterprise-auth's main.go: assigning a nil
*oidc.Provider to the handler's interface field would have produced a
non-nil interface wrapping a nil pointer (Go's classic typed-nil trap),
silently breaking the "OIDC not configured" no-op path -- New() now
takes the concrete pointer type and checks it before ever converting to
the interface, with a regression test pinning the fix down.

Still missing: SAML's equivalent (ACS endpoint), a tenant-picker UI for
multi-membership identities, and any admin UI to actually create a
tenant_memberships row (today that's manual SQL, documented in the
runbook's new bootstrap walkthrough).
 2026-08-13 23:00:35 -07:00
jcoffey-dev 1d57e697b1 Phase 4: real per-tenant ClickHouse isolation via a new enterprise-api binary 
Closes the threat model's headline finding for the SQL query path:
enterprise/internal/tenantprovision does real CREATE DATABASE/USER/GRANT
against ClickHouse, and enterprise/internal/chrunner is a per-tenant
connection registry implementing api's SQLRunner interface, resolving
the tenant from the authenticated request identity -- never a
caller-suppliable parameter. Both are wired into a new binary,
enterprise/cmd/enterprise-api, alongside the unchanged single-tenant
api/cmd/api, since AGPL core can never import enterprise/ and Go's own
internal/ package visibility rules meant enterprise/ couldn't implement
core's SQLRunner interface without importing the package that defines
it. That required moving api/internal/{authz,queryapi,dashboards,
querylang/executor,searchclient,httpserver} out of internal/ -- the
minimal set enterprise-api needs to import; querylang's compiler
internals (planner/lexer/parser/ast/ir) and api's own config stay
internal, since nothing outside api needs them directly.

Also finally wires enterprise/internal/audit into queryapi.AuditLogger
(nil since Phase 4 task 4) via a new adapter, and adds live-ClickHouse
integration tests for two of the four adversarial probes named in
docs/phase-4-isolation-design.md's verification plan.

Corrected several overclaims in the docs while writing this up: an
earlier claim that rbacstore's CRUD was "verified against a live
Postgres" was never actually true in this environment (only
internal/audit was, earlier in this phase, before Docker access was
lost) -- threat-model.md, phase-4-runbook.md, CLAUDE.md, and
enterprise/README.md all now distinguish "a real integration test
exists" from "this was confirmed against a live database."

Still not built: Tantivy/free-text tenant isolation
(enterprise/internal/searchclient), and any deployment-topology
mechanism that actually routes traffic to enterprise-api instead of
plain api -- both binaries exist side by side today with nothing
enforcing or flagging which one a deployment runs.
 2026-08-13 22:48:38 -07:00
jcoffey-dev 3eb0f4c589 Phase 4: SSO scaffolding, RBAC enforcement, tenant-scoped dashboards, audit logging, K8s deployment 
RBAC (api/internal/authz) is live on /query and /dashboards, backed by a
new enterprise/ module (session issuance, audit logging, RBAC storage,
OIDC/SAML protocol wiring) that core never imports -- only calls over
HTTP. Found and fixed a real cross-tenant vulnerability in dashboards
(no tenant_id filtering at all) while writing the threat model doc.

Two things are explicitly NOT done, documented rather than hidden:
tenant isolation for log data itself (/query still shares one ClickHouse
connection and Tantivy index across every tenant -- RBAC controls who
can query, not what a query can see), and human SSO login (protocol
wiring exists, no HTTP handler calls it yet). See
docs/security/threat-model.md and docs/phase-4-runbook.md.

Also adds deploy/ (Go Operator + Helm chart, validated offline only --
no cluster was reachable in this environment).
 2026-08-13 22:16:59 -07:00