Closes a gap named across CLAUDE.md/docs/architecture.md/deploy/README.md
since early Phase 4: the operator's Tenant CRD and -provision-tenant
were two disconnected mechanisms. The operator's reconciler generated a
K8s Secret with a locally-generated random password that authenticated
against nothing (nothing ever called ClickHouse to create a matching
user), and unconditionally claimed status.phase=Active the moment a
Tenant object existed -- actively misleading, not just incomplete.
Two unification shapes were considered (surfaced to the user via
AskUserQuestion, given the real difference in blast radius): the
operator's reconcile loop becoming a second real actor (new Postgres +
ClickHouse admin credentials flowing into the K8s controller, plus real
reconcile-loop idempotency/retry design for an inherently one-shot
external side effect), or keeping -provision-tenant as the sole real
actor and having it also sync its result into the CRD. Went with the
lighter option.
enterprise/internal/tenantcrd (new): a Syncer using the K8s dynamic
client (unstructured.Unstructured + a GroupVersionResource, not
deploy/operator's typed Tenant struct -- avoids a cross-module Go
dependency between two independently-versioned modules for one type).
Upserts the Tenant object, creates/updates a Secret with the *real*
ClickHouse credentials owned by that Tenant via an OwnerReference, then
patches status.{clickHouseDatabaseName,clickHouseSecretRef,
tantivyIndexPath}. Idempotent and safe to retry: never rotates a
credential across a re-sync, never overwrites a pre-existing
spec.displayName a human/GitOps process set.
cmd/enterprise-api/main.go's runProvisionTenant calls Sync when
TENANT_CRD_NAMESPACE is set (empty = no-op, same shape as every other
optional dependency in this codebase). Its "already active" refusal is
now split: ClickHouse re-provisioning is still refused (rotating a live
credential would break every open connection for no benefit), but CR
sync alone is now retryable using the credentials already on file in
rbacstore -- needed for retrying a previously-failed sync, or
backfilling CR sync for a tenant provisioned before this existed.
deploy/operator's reconciler rewritten to match: it never claims
PhaseActive on its own initiative anymore, only once
status.ClickHouseDatabaseName is non-empty (the field -provision-tenant,
and only -provision-tenant, sets). Phase is now a pure function of
{spec.suspended, status.ClickHouseDatabaseName != ""} recomputed every
reconcile, not toggled in place -- fixes a related bug the old code
would have hit once suspension was involved: un-suspending an
already-provisioned tenant needs to return straight to Active, which
isn't derivable from "last observed phase was Suspended" alone. The
reconciler no longer creates or manages any Secret, dropped its
`secrets` RBAC grant entirely, and gained zero new dependencies.
Helm chart: enterprise-api gets its own ServiceAccount/Role/RoleBinding
(get/list/create tenants, get/update/patch tenants/status, get/create/
update secrets -- least-privilege, scoped to the release namespace, not
a ClusterRole) and a TENANT_CRD_NAMESPACE env var, both gated on
tenantOperator.enabled. tenant-operator's ClusterRole loses the
secrets grant it no longer needs.
Verified in this environment: enterprise/internal/tenantcrd's tests run
against k8s.io/client-go's fake dynamic + typed clientsets (real client
library, fake transport, no cluster needed); deploy/operator's rewritten
tenant_controller_test.go runs against controller-runtime's fake
client, including new regression tests for the "must not claim Active
without confirmation" and "un-suspending returns to Active, not
Provisioning" properties; helm template + parsing the rendered YAML
confirms the RBAC split renders exactly as designed under both
tenantOperator.enabled=true/false. Not verified: an actual
-provision-tenant run against a real cluster with the operator watching
(no live cluster in this environment, same disclosed limitation as the
rest of /deploy). Docs updated in lockstep: CLAUDE.md, docs/architecture.md,
deploy/README.md, deploy/helm/sentry/README.md (including a corrected
"Trying the two-tenant example" walkthrough), phase-4-runbook.md (new
§11), enterprise/README.md. Also fixed two unrelated stale claims found
along the way: docs/architecture.md still said docker-compose.yml ran
plain api unconditionally (fixed in an earlier commit, doc not updated
then), and enterprise-api's own main.go doc comment still said Helm/
docker-compose wiring wasn't built yet.
173 lines
6.9 KiB
YAML
173 lines
6.9 KiB
YAML
{{/*
|
|
Mutually exclusive with api.yaml's Deployment+Service -- see that file's
|
|
doc comment. This is the concrete fix for the deployment-topology gap
|
|
/docs/security/threat-model.md names as the single largest remaining
|
|
Phase 4 issue once both storage engines' isolation mechanisms were
|
|
built: "nothing forces or flags whether a deployment runs the isolated
|
|
binary." With this file, it's not a separate knob to forget -- the same
|
|
enterprise.enabled that turns on RBAC/audit/SSO also swaps which query
|
|
binary actually serves traffic. Uses the "api" selector label (not
|
|
"enterprise-api") deliberately, so the shared Service name+port below
|
|
routes to whichever Deployment is actually rendered, with zero
|
|
conditional logic needed in any consumer (alerting, web).
|
|
*/}}
|
|
{{- if .Values.enterprise.enabled }}
|
|
{{- if .Values.tenantOperator.enabled }}
|
|
# Grants enterprise-api's -provision-tenant (enterprise/internal/
|
|
# tenantcrd) permission to sync real provisioning results into the
|
|
# Tenant CRD -- a Role, not a ClusterRole (unlike tenant-operator's:
|
|
# this binary only ever provisions tenants that live in its own release
|
|
# namespace, no reason to widen it), scoped to exactly the two resource
|
|
# types tenantcrd.Syncer touches. Only rendered when tenantOperator is
|
|
# also enabled -- no Tenant CRD installed, nothing to sync into.
|
|
apiVersion: v1
|
|
kind: ServiceAccount
|
|
metadata:
|
|
name: {{ .Release.Name }}-enterprise-api
|
|
labels:
|
|
{{- include "sentry.labels" . | nindent 4 }}
|
|
---
|
|
apiVersion: rbac.authorization.k8s.io/v1
|
|
kind: Role
|
|
metadata:
|
|
name: {{ .Release.Name }}-enterprise-api
|
|
labels:
|
|
{{- include "sentry.labels" . | nindent 4 }}
|
|
rules:
|
|
- apiGroups: ["sentry.io"]
|
|
resources: ["tenants"]
|
|
verbs: ["get", "list", "create"]
|
|
- apiGroups: ["sentry.io"]
|
|
resources: ["tenants/status"]
|
|
verbs: ["get", "update", "patch"]
|
|
- apiGroups: [""]
|
|
resources: ["secrets"]
|
|
verbs: ["get", "create", "update"]
|
|
---
|
|
apiVersion: rbac.authorization.k8s.io/v1
|
|
kind: RoleBinding
|
|
metadata:
|
|
name: {{ .Release.Name }}-enterprise-api
|
|
labels:
|
|
{{- include "sentry.labels" . | nindent 4 }}
|
|
roleRef:
|
|
apiGroup: rbac.authorization.k8s.io
|
|
kind: Role
|
|
name: {{ .Release.Name }}-enterprise-api
|
|
subjects:
|
|
- kind: ServiceAccount
|
|
name: {{ .Release.Name }}-enterprise-api
|
|
namespace: {{ .Release.Namespace }}
|
|
---
|
|
{{- end }}
|
|
apiVersion: apps/v1
|
|
kind: Deployment
|
|
metadata:
|
|
name: {{ .Release.Name }}-api
|
|
labels:
|
|
{{- include "sentry.labels" . | nindent 4 }}
|
|
{{- include "sentry.selectorLabels" (list $ "api") | nindent 4 }}
|
|
app.kubernetes.io/component: enterprise-api
|
|
spec:
|
|
replicas: {{ .Values.api.replicas }}
|
|
selector:
|
|
matchLabels:
|
|
{{- include "sentry.selectorLabels" (list $ "api") | nindent 6 }}
|
|
template:
|
|
metadata:
|
|
labels:
|
|
{{- include "sentry.selectorLabels" (list $ "api") | nindent 8 }}
|
|
spec:
|
|
{{- if .Values.tenantOperator.enabled }}
|
|
serviceAccountName: {{ .Release.Name }}-enterprise-api
|
|
{{- end }}
|
|
initContainers:
|
|
{{- include "sentry.waitForTCP" (list "clickhouse" (printf "%s-clickhouse" .Release.Name) "9000") | nindent 8 }}
|
|
{{- include "sentry.waitForTCP" (list "postgres" (printf "%s-postgres" .Release.Name) "5432") | nindent 8 }}
|
|
{{- include "sentry.waitForTCP" (list "search" (printf "%s-search" .Release.Name) "50052") | nindent 8 }}
|
|
{{- include "sentry.waitForTCP" (list "enterprise-auth" (printf "%s-enterprise-auth" .Release.Name) "8082") | nindent 8 }}
|
|
containers:
|
|
- name: enterprise-api
|
|
image: "{{ .Values.enterprise.apiImage.repository }}:{{ .Values.enterprise.apiImage.tag }}"
|
|
imagePullPolicy: {{ .Values.global.imagePullPolicy }}
|
|
env:
|
|
# :8080, not enterprise-api's own :8083 default -- this
|
|
# container occupies the same Service/port every consumer
|
|
# (alerting's API_QUERY_URL, web's build args) already
|
|
# expects "-api:8080" to mean. See this file's doc comment.
|
|
- name: HTTP_LISTEN_ADDR
|
|
value: ":8080"
|
|
- name: CLICKHOUSE_ADDR
|
|
value: "{{ .Release.Name }}-clickhouse:9000"
|
|
# tenantprovision's admin connection -- the same credential
|
|
# clickhouse-migrate uses, needs access_management, never a
|
|
# tenant-scoped grant. See enterprise/internal/tenantprovision's
|
|
# doc comment.
|
|
- name: CLICKHOUSE_ADMIN_USERNAME
|
|
value: "default"
|
|
- name: CLICKHOUSE_ADMIN_PASSWORD
|
|
valueFrom:
|
|
secretKeyRef:
|
|
name: {{ .Release.Name }}-clickhouse
|
|
key: password
|
|
- name: SEARCH_GRPC_ADDR
|
|
value: "{{ .Release.Name }}-search:50052"
|
|
- name: POSTGRES_ADDR
|
|
value: "{{ .Release.Name }}-postgres:5432"
|
|
- name: POSTGRES_DATABASE
|
|
value: sentry_metadata
|
|
- name: POSTGRES_USERNAME
|
|
value: sentry
|
|
- name: POSTGRES_PASSWORD
|
|
valueFrom:
|
|
secretKeyRef:
|
|
name: {{ .Release.Name }}-postgres
|
|
key: password
|
|
# Restricted audit_writer Postgres role (Phase 4 task 4) --
|
|
# its own pool, never the shared "sentry" credential above.
|
|
# See enterprise/internal/audit's doc comment.
|
|
- name: AUDIT_WRITER_USERNAME
|
|
value: "audit_writer"
|
|
- name: AUDIT_WRITER_PASSWORD
|
|
valueFrom:
|
|
secretKeyRef:
|
|
name: {{ .Release.Name }}-postgres
|
|
key: auditWriterPassword
|
|
- name: ENTERPRISE_AUTH_URL
|
|
value: "http://{{ .Release.Name }}-enterprise-auth:8082"
|
|
{{- if .Values.tenantOperator.enabled }}
|
|
# Enables enterprise/internal/tenantcrd -- -provision-tenant
|
|
# (run via `kubectl exec` into this Deployment's Pod, using
|
|
# its ServiceAccount/Role above) syncs real provisioning
|
|
# results into the Tenant CRD this namespace's tenants live
|
|
# in. Unset (the default, when tenantOperator isn't enabled)
|
|
# is a documented no-op -- see apiconfig.Config.TenantCRDNamespace.
|
|
- name: TENANT_CRD_NAMESPACE
|
|
value: {{ .Release.Namespace | quote }}
|
|
{{- end }}
|
|
ports:
|
|
- name: http
|
|
containerPort: 8080
|
|
readinessProbe:
|
|
exec:
|
|
command: ["/enterprise-api", "-healthcheck"]
|
|
initialDelaySeconds: 5
|
|
periodSeconds: 5
|
|
resources:
|
|
{{- toYaml .Values.api.resources | nindent 12 }}
|
|
---
|
|
apiVersion: v1
|
|
kind: Service
|
|
metadata:
|
|
name: {{ .Release.Name }}-api
|
|
labels:
|
|
{{- include "sentry.labels" . | nindent 4 }}
|
|
{{- include "sentry.selectorLabels" (list $ "api") | nindent 4 }}
|
|
spec:
|
|
selector:
|
|
{{- include "sentry.selectorLabels" (list $ "api") | nindent 4 }}
|
|
ports:
|
|
- name: http
|
|
port: 8080
|
|
{{- end }}
|