jcoffey-dev
17fdc212c2
Ingest tenant-awareness was named "undesigned, not just unbuilt" across CLAUDE.md/threat-model.md/the runbook since early Phase 4 -- the last major standing gap. Scoping was agreed via AskUserQuestion: a config-supplied tenant_id + shared-secret token ingest validates (smaller real implementation, no new PKI), over per-tenant mTLS certs. This change builds that identity mechanism end to end and attaches it to every record at the point it enters the system; it deliberately does NOT build per-tenant write-routing for ClickHouse or Tantivy -- that's real, separately-scoped follow-up work, disclosed explicitly everywhere this was previously called undesigned, not silently left half-done. New pieces: - metadata/migrations/0034 + enterprise/internal/rbacstore/ ingest_credentials.go: a per-tenant bearer credential, only its SHA-256 hash ever persisted (same reasoning a password gets hashed, not stored raw) -- CreateIngestCredential returns the plaintext exactly once, ValidateIngestCredential/RevokeIngestCredential/ ListIngestCredentialsForTenant round it out. - enterprise-auth gains -create-ingest-credential-tenant/ -list-ingest-credentials-tenant/-revoke-ingest-credential (same offline-operator-flag shape as every other credential-minting flag in this binary) and a new POST /internal/authorize-ingest endpoint (internal/authhandler) validating a presented token and resolving its tenant -- a genuinely different credential type from session-backed /internal/authorize, so it doesn't touch session.Manager at all. - ingest (AGPL core) gains an optional TenantResolver (internal/grpcserver, nil by default) and its HTTP client implementation (internal/tenantresolver.HTTPResolver) -- a plain HTTP call to enterprise-auth's new endpoint, never an enterprise/ import, same "network boundary, not import boundary" shape api/authz.HTTPAuthorizer already uses for the query path. PushBatch now requires an `authorization: Bearer <token>` gRPC metadata entry once a resolver is configured, fails the whole batch closed on a missing/invalid credential (never falls back to "no tenant"), and attaches the resolved tenant ID to every record as a `tenant_id` Kafka message header before producing it. Verified with real round trips at every layer, no Docker needed: rbacstore's credential CRUD (skip-gated on live Postgres, same as every other rbacstore integration test this phase), authhandler's new endpoint (real HTTP via httptest, including the regression test that a session token must not validate as an ingest credential), tenantresolver (real HTTP client against httptest, same pattern as authz.HTTPAuthorizer's own tests), and grpcserver's PushBatch (fake resolver/producer -- no resolver leaves messages unchanged, a configured resolver attaches the right header or fails closed on a bad/missing token). Helm: ingest.requireTenantCredential (default false) is a deliberate, separate opt-in from enterprise.enabled -- turning ENTERPRISE_AUTH_URL on for ingest requires every agent to already hold a credential or be refused outright, so it must not default on just because enterprise.enabled does (same reasoning api.yaml's ENTERPRISE_AUTH_URL isn't tied to enterprise.enabled directly either). docker-compose.yml leaves it unset, same as ever. Docs updated everywhere this was called "undesigned": CLAUDE.md, docs/architecture.md, docs/security/threat-model.md (including its summary table, now split into "identity: built" vs "write-routing: not yet"), docs/phase-4-runbook.md (new §13), enterprise/README.md.
84 lines
3.2 KiB
YAML
84 lines
3.2 KiB
YAML
apiVersion: apps/v1
|
|
kind: Deployment
|
|
metadata:
|
|
name: {{ .Release.Name }}-ingest
|
|
labels:
|
|
{{- include "sentry.labels" . | nindent 4 }}
|
|
{{- include "sentry.selectorLabels" (list $ "ingest") | nindent 4 }}
|
|
spec:
|
|
replicas: {{ .Values.ingest.replicas }}
|
|
selector:
|
|
matchLabels:
|
|
{{- include "sentry.selectorLabels" (list $ "ingest") | nindent 6 }}
|
|
template:
|
|
metadata:
|
|
labels:
|
|
{{- include "sentry.selectorLabels" (list $ "ingest") | nindent 8 }}
|
|
spec:
|
|
initContainers:
|
|
{{- include "sentry.waitForTCP" (list "redpanda" (printf "%s-redpanda" .Release.Name) "9092") | nindent 8 }}
|
|
{{- include "sentry.waitForTCP" (list "clickhouse" (printf "%s-clickhouse" .Release.Name) "9000") | nindent 8 }}
|
|
containers:
|
|
- name: ingest
|
|
image: "{{ .Values.ingest.image.repository }}:{{ .Values.ingest.image.tag }}"
|
|
imagePullPolicy: {{ .Values.global.imagePullPolicy }}
|
|
env:
|
|
- name: REDPANDA_BROKERS
|
|
value: "{{ .Release.Name }}-redpanda:9092"
|
|
- name: CLICKHOUSE_ADDR
|
|
value: "{{ .Release.Name }}-clickhouse:9000"
|
|
- name: CLICKHOUSE_PASSWORD
|
|
valueFrom:
|
|
secretKeyRef:
|
|
name: {{ .Release.Name }}-clickhouse
|
|
key: password
|
|
{{- if and .Values.enterprise.enabled .Values.ingest.requireTenantCredential }}
|
|
# Enables ingest/internal/grpcserver.TenantResolver.
|
|
# Deliberately its OWN opt-in, not folded into
|
|
# enterprise.enabled directly (same reasoning
|
|
# api.yaml/enterprise-api.yaml's ENTERPRISE_AUTH_URL isn't
|
|
# set just because enterprise.enabled is true -- see that
|
|
# env var's own comment there): turning this on requires
|
|
# every agent to already present a valid `Authorization:
|
|
# Bearer <ingest token>` (minted via `enterprise-auth
|
|
# -create-ingest-credential-tenant=<id>`) or be refused
|
|
# outright, which would silently break ingest for any
|
|
# not-yet-reconfigured agent if it defaulted on alongside
|
|
# enterprise.enabled. Off (the default) leaves every record
|
|
# without a tenant_id header, same as every Phase 0-3
|
|
# deployment.
|
|
- name: ENTERPRISE_AUTH_URL
|
|
value: "http://{{ .Release.Name }}-enterprise-auth:8082"
|
|
{{- end }}
|
|
ports:
|
|
- name: grpc
|
|
containerPort: 4317
|
|
{{- if .Values.ingest.tlsSecretName }}
|
|
volumeMounts:
|
|
- name: tls
|
|
mountPath: /etc/sentry-ingest
|
|
readOnly: true
|
|
{{- end }}
|
|
resources:
|
|
{{- toYaml .Values.ingest.resources | nindent 12 }}
|
|
{{- if .Values.ingest.tlsSecretName }}
|
|
volumes:
|
|
- name: tls
|
|
secret:
|
|
secretName: {{ .Values.ingest.tlsSecretName }}
|
|
{{- end }}
|
|
---
|
|
apiVersion: v1
|
|
kind: Service
|
|
metadata:
|
|
name: {{ .Release.Name }}-ingest
|
|
labels:
|
|
{{- include "sentry.labels" . | nindent 4 }}
|
|
{{- include "sentry.selectorLabels" (list $ "ingest") | nindent 4 }}
|
|
spec:
|
|
selector:
|
|
{{- include "sentry.selectorLabels" (list $ "ingest") | nindent 4 }}
|
|
ports:
|
|
- name: grpc
|
|
port: 4317
|