jcoffey-dev
ba2276aa1a
Closes the last named "isolation mechanism" gap: search.proto gains a tenant_id field on SearchRequest; search/src/registry.rs's IndexRegistry resolves it to an on-demand-opened, per-tenant Tantivy index (empty tenant_id keeps today's single default index, so this is purely additive); enterprise/internal/searchclient sets that field from the authenticated request identity in ctx, mirroring chrunner's exact fail-closed "never a parameter" shape. Wired into enterprise-api in place of the shared api/searchclient. Unlike the ClickHouse pieces from the previous two commits, this one is genuinely verified end to end in this environment: Tantivy is an embedded library, not a networked service, so both the Rust index registry (cargo test, cargo clippy --all-targets -- -D warnings, both clean) and the Go client (a real in-process gRPC server) could actually run. registry.rs's tenant_index_is_isolated_from_default_and_other_tenants seeds three real indices with the same term and confirms a tenant-scoped search returns only that tenant's document -- item 3 of the isolation design doc's verification plan, closed for real, not just written. With both ClickHouse and Tantivy isolation now built, the single largest remaining gap is no longer a missing mechanism: it's that nothing forces or flags whether a deployment actually runs enterprise-api instead of plain api, and that ingest itself has no tenant concept for either storage engine (every record still lands in the one shared database/ index no matter what -- undesigned, not just unbuilt). Updated the threat model, architecture doc, CLAUDE.md, and both READMEs accordingly.
api
Sentry's query API: a single POST /query endpoint accepting either the
pipe syntax or raw SQL, compiled and routed across ClickHouse and Tantivy
by internal/querylang. Replaces Phase 0/1's two separate placeholder
endpoints (raw-SQL-only /query, free-text-only /search) — see
/docs/query-language-design.md for the grammar, IR, and routing design,
and /docs/query-language-reference.md for the user-facing syntax.
Why plain REST, not gRPC + REST gateway
CLAUDE.md pins the control plane to "Go, gRPC + REST gateway." This
service is plain net/http instead — a deliberate simplification, not a
change to the pinned stack. Wiring up a .proto service,
google.api.http annotations, and protoc-gen-grpc-gateway codegen for
one endpoint doesn't buy much at this size. api does speak gRPC
internally — to /search — this simplification is about the
public-facing surface only.
Endpoint
POST /query — body {"query": "...", "language": ""}, response
{"columns": [...], "rows": [[...], ...]} or {"error": "..."}.
queryis either pipe syntax (service=api | where status>=500 | stats count by host) or raw SQL (SELECT ...). Auto-detected by whether the query starts withSELECT(case-insensitive).languageoptionally overrides detection:"sql"or"spl". Exists for the rare case a pipe query legitimately starts with the literal word "select" as a bare search term.- Both syntaxes compile to the same
querylang/ir.Planand execute through the same code path — seeinternal/querylang/executorfor the four routing cases (pure ClickHouse; Tantivy prefilter + ClickHouse rows; Tantivy prefilter + ClickHouse aggregation; raw SQL passthrough).
GET /healthz — for docker-compose/k8s liveness checks.
No auth. Not scoped yet — don't expose this beyond a trusted dev/homelab network.
Configuration
Environment variables (see internal/config/config.go):
| Var | Default | Purpose |
|---|---|---|
HTTP_LISTEN_ADDR |
:8080 |
|
CLICKHOUSE_ADDR |
localhost:9000 |
Native protocol port |
CLICKHOUSE_DATABASE / _USERNAME / _PASSWORD |
sentry / default / `` |
|
SEARCH_GRPC_ADDR |
localhost:50052 |
Must match /search's GRPC_LISTEN_ADDR |
QUERY_TIMEOUT_SECONDS |
30 |
Per-request timeout |
CORS_ALLOWED_ORIGIN |
* |
Wide open by default since there's no auth yet; tighten together |
searchclient.Dial connects to /search over plain TCP, no TLS — same
trust boundary as api's existing plain-TCP connection to ClickHouse.
mTLS in this project is specifically the agent↔ingest edge boundary, not
every internal hop.
Building & testing
go build ./...
go vet ./...
go test ./...
# from the repo root, not api/
docker build -f api/Dockerfile -t sentry-api .
Testing notes
internal/queryapi's HTTP handler depends on ClickHouse and /search
only through the narrow interfaces querylang/executor defines
(SQLRunner, SearchClient), so routing, compilation, JSON encoding,
and error-status mapping are all unit-tested against fakes — no live
ClickHouse or /search instance needed, and the real lexer/parser/
planner run unmocked in these tests, only the backends are faked. See
internal/querylang's own package docs for how compilation and
execution are tested independently of each other. executor.ChRunner
(the reflection-based row scanning against ClickHouse's driver.Rows)
and internal/searchclient's actual gRPC dial are not unit-tested — the
former because faking driver.Rows fully would be significant
test-only scaffolding the driver's own docs say isn't meant to be
implemented by adopters; the latter because it's a thin wrapper with
nothing but wiring to test. Both are exercised end-to-end via the
docker-compose flow in /docs/phase-2-runbook.md.