jcoffey-dev
13cf9a30cb
Full rebrand across cosmetic branding, code identifiers, and infrastructure/data-plane naming, using the supplied Cairn OBS logo package. Cosmetic: favicon/logo swap (also closes a stale license-audit finding -- the old favicon was SvelteKit's unreplaced scaffold logo), new centered welcome landing page, larger/legible sidebar logo, page titles, CLAUDE.md/README/docs prose. Code identifiers: Go module path github.com/sentry/sentry -> github.com/cairnobs/cairnobs across all 13 modules and ~91 files (protoc regenerated); Rust crates sentry-agent/sentry-parser/sentry-search -> cairnobs-*; CLI sentryctl -> cairnobsctl; Terraform provider fully renamed (sentry_dashboard etc. -> cairnobs_dashboard, provider type, env vars); every session/auth cookie name; agent config paths and Windows service identity. Deliberately preserved: the gRPC wire protocol's protobuf packages (sentry.logs.v1, sentry.agent.v1) and their Go import directory (proto/sentry/...) -- renaming the wire-level package would break every currently-deployed agent binary (confirmed two real hosts, including mail.inbuxa.com, are actively streaming through this exact contract) until rebuilt and redeployed in lockstep with an ingest cutover. Only the Go module path wrapping the generated code changes. Infrastructure: every docker-compose container name (root and three component-level compose files); the Helm chart (directory, Chart.yaml, named-template helpers, all templates, values.yaml image repos); Kubernetes Operator (CRD group sentry.io -> cairnobs.io, both CRD YAML files, Go identifiers, RBAC markers); the coupled enterprise/tenantcrd package. Caught and fixed real path-coupling bugs along the way: the Helm chart's search/ingest volume mounts and the dev-only-credential detection constant vs. docker-compose.yml's literal values had to move together or a security warning would have silently stopped firing. Data plane: Postgres database sentry_metadata -> cairnobs_metadata and role sentry -> cairnobs; ClickHouse database sentry -> cairnobs; Kafka topic sentry.logs.raw -> cairnobs.logs.raw and its consumer groups. Source-level defaults, docker-compose.yml, and every migrate.sh/ provision script default updated together; already-applied migration files left untouched per this repo's immutable-migration convention. Verified at every layer: all 13 Go modules build/vet/test clean, both Rust workspaces (agent, search) build/clippy/test clean, npm run check/ build clean, docker compose config validates on all four compose files. Live-verified against a real docker stack multiple times through this work, including a final fresh-volume run confirming the actual renamed Postgres database/role, ClickHouse database, and Kafka topic all work end to end with a real login and query, zero console errors.
73 lines
2.8 KiB
Go
73 lines
2.8 KiB
Go
// Package executor runs a compiled ir.Plan and returns results in a
|
|
// shape consistent regardless of which backend(s) were hit -- the point
|
|
// of compiling to one IR in the first place. See
|
|
// /docs/query-language-design.md's "Execution" section for the four
|
|
// routing cases implemented here.
|
|
package executor
|
|
|
|
import (
|
|
"context"
|
|
"fmt"
|
|
|
|
"github.com/cairnobs/cairnobs/api/internal/querylang/ir"
|
|
)
|
|
|
|
type Result struct {
|
|
Columns []string
|
|
Rows [][]any
|
|
}
|
|
|
|
// SQLRunner executes a raw SQL statement against ClickHouse. *ChRunner
|
|
// (chrunner.go) is the production implementation; tests use a fake --
|
|
// same narrow-interface pattern used throughout /ingest and /api.
|
|
type SQLRunner interface {
|
|
RunSQL(ctx context.Context, sql string) (*Result, error)
|
|
}
|
|
|
|
// SearchClient resolves a Tantivy query into matching record_ids.
|
|
type SearchClient interface {
|
|
Search(ctx context.Context, query string, limit uint32) ([]string, error)
|
|
}
|
|
|
|
// textSearchLimit caps how many record_ids a Tantivy prefilter can feed
|
|
// into a ClickHouse `IN (...)` clause. See /docs/query-language-design.md's
|
|
// "Known scaling limitation" -- this is a real, disclosed limit on result
|
|
// completeness for very broad text searches, not an oversight.
|
|
//
|
|
// 5000, not 10000: confirmed by actually running the Phase 2 benchmark
|
|
// (see /docs/phase-2-runbook.md) that 10000 quoted UUIDs (~39 bytes each
|
|
// including the comma) produces a ~390KB query string, which exceeds
|
|
// ClickHouse's default max_query_size (262144 bytes / 256KiB) and fails
|
|
// outright with a syntax error rather than degrading gracefully. 5000
|
|
// UUIDs is ~195KB, safely under that default with headroom for the rest
|
|
// of the query. This was a real failure caught by running the benchmark,
|
|
// not a value chosen from first-principles estimation.
|
|
const textSearchLimit = 5000
|
|
|
|
// Execute runs plan against the given backends. The four cases (per the
|
|
// design doc): RawSQL passthrough; pure ClickHouse (no TextSearch); text
|
|
// search alone (Tantivy prefilter -> ClickHouse row fetch); text search
|
|
// plus aggregation (Tantivy prefilter -> ClickHouse aggregate). Cases 2-4
|
|
// share the same buildSQL/buildWhereClause code (sql.go) -- the only
|
|
// difference is whether a record_id filter is threaded in.
|
|
func Execute(ctx context.Context, plan *ir.Plan, sqlRunner SQLRunner, search SearchClient) (*Result, error) {
|
|
if plan.RawSQL != "" {
|
|
return sqlRunner.RunSQL(ctx, plan.RawSQL)
|
|
}
|
|
|
|
var recordIDFilter []string
|
|
if len(plan.TextSearch) > 0 {
|
|
ids, err := search.Search(ctx, plan.TextSearch[0].Query, textSearchLimit)
|
|
if err != nil {
|
|
return nil, fmt.Errorf("full-text search failed: %w", err)
|
|
}
|
|
if len(ids) == 0 {
|
|
return &Result{Columns: []string{}, Rows: [][]any{}}, nil
|
|
}
|
|
recordIDFilter = ids
|
|
}
|
|
|
|
sql := buildSQL(plan, recordIDFilter)
|
|
return sqlRunner.RunSQL(ctx, sql)
|
|
}
|