Unify the Tenant CRD with enterprise-api -provision-tenant (lightweight)
Closes a gap named across CLAUDE.md/docs/architecture.md/deploy/README.md
since early Phase 4: the operator's Tenant CRD and -provision-tenant
were two disconnected mechanisms. The operator's reconciler generated a
K8s Secret with a locally-generated random password that authenticated
against nothing (nothing ever called ClickHouse to create a matching
user), and unconditionally claimed status.phase=Active the moment a
Tenant object existed -- actively misleading, not just incomplete.
Two unification shapes were considered (surfaced to the user via
AskUserQuestion, given the real difference in blast radius): the
operator's reconcile loop becoming a second real actor (new Postgres +
ClickHouse admin credentials flowing into the K8s controller, plus real
reconcile-loop idempotency/retry design for an inherently one-shot
external side effect), or keeping -provision-tenant as the sole real
actor and having it also sync its result into the CRD. Went with the
lighter option.
enterprise/internal/tenantcrd (new): a Syncer using the K8s dynamic
client (unstructured.Unstructured + a GroupVersionResource, not
deploy/operator's typed Tenant struct -- avoids a cross-module Go
dependency between two independently-versioned modules for one type).
Upserts the Tenant object, creates/updates a Secret with the *real*
ClickHouse credentials owned by that Tenant via an OwnerReference, then
patches status.{clickHouseDatabaseName,clickHouseSecretRef,
tantivyIndexPath}. Idempotent and safe to retry: never rotates a
credential across a re-sync, never overwrites a pre-existing
spec.displayName a human/GitOps process set.
cmd/enterprise-api/main.go's runProvisionTenant calls Sync when
TENANT_CRD_NAMESPACE is set (empty = no-op, same shape as every other
optional dependency in this codebase). Its "already active" refusal is
now split: ClickHouse re-provisioning is still refused (rotating a live
credential would break every open connection for no benefit), but CR
sync alone is now retryable using the credentials already on file in
rbacstore -- needed for retrying a previously-failed sync, or
backfilling CR sync for a tenant provisioned before this existed.
deploy/operator's reconciler rewritten to match: it never claims
PhaseActive on its own initiative anymore, only once
status.ClickHouseDatabaseName is non-empty (the field -provision-tenant,
and only -provision-tenant, sets). Phase is now a pure function of
{spec.suspended, status.ClickHouseDatabaseName != ""} recomputed every
reconcile, not toggled in place -- fixes a related bug the old code
would have hit once suspension was involved: un-suspending an
already-provisioned tenant needs to return straight to Active, which
isn't derivable from "last observed phase was Suspended" alone. The
reconciler no longer creates or manages any Secret, dropped its
`secrets` RBAC grant entirely, and gained zero new dependencies.
Helm chart: enterprise-api gets its own ServiceAccount/Role/RoleBinding
(get/list/create tenants, get/update/patch tenants/status, get/create/
update secrets -- least-privilege, scoped to the release namespace, not
a ClusterRole) and a TENANT_CRD_NAMESPACE env var, both gated on
tenantOperator.enabled. tenant-operator's ClusterRole loses the
secrets grant it no longer needs.
Verified in this environment: enterprise/internal/tenantcrd's tests run
against k8s.io/client-go's fake dynamic + typed clientsets (real client
library, fake transport, no cluster needed); deploy/operator's rewritten
tenant_controller_test.go runs against controller-runtime's fake
client, including new regression tests for the "must not claim Active
without confirmation" and "un-suspending returns to Active, not
Provisioning" properties; helm template + parsing the rendered YAML
confirms the RBAC split renders exactly as designed under both
tenantOperator.enabled=true/false. Not verified: an actual
-provision-tenant run against a real cluster with the operator watching
(no live cluster in this environment, same disclosed limitation as the
rest of /deploy). Docs updated in lockstep: CLAUDE.md, docs/architecture.md,
deploy/README.md, deploy/helm/sentry/README.md (including a corrected
"Trying the two-tenant example" walkthrough), phase-4-runbook.md (new
§11), enterprise/README.md. Also fixed two unrelated stale claims found
along the way: docs/architecture.md still said docker-compose.yml ran
plain api unconditionally (fixed in an earlier commit, doc not updated
then), and enterprise-api's own main.go doc comment still said Helm/
docker-compose wiring wasn't built yet.
This commit is contained in:
+22
-16
@@ -146,23 +146,29 @@ escape hatch is opaque to any compiler-injected filter.
|
||||
query time — and permanently empty until something upstream of
|
||||
`chrunner`/`searchclient` becomes tenant-aware on the write side,
|
||||
which is undesigned, not merely unbuilt.
|
||||
- `deploy/operator`'s `Tenant` CRD still manages only the K8s-side
|
||||
artifact (a credential Secret); it doesn't call
|
||||
`enterprise-api -provision-tenant` or otherwise trigger ClickHouse-side
|
||||
provisioning. The two mechanisms are independent today, not reconciled
|
||||
into one state machine.
|
||||
- `deploy/operator`'s `Tenant` CRD and `enterprise-api -provision-tenant`
|
||||
are now unified, deliberately lightweight: `-provision-tenant` stays
|
||||
the sole real actor (ClickHouse + `rbacstore`), and now also syncs its
|
||||
real result into the `Tenant` CRD (`enterprise/internal/tenantcrd`) --
|
||||
a real credential Secret, and status fields the reconciler
|
||||
(`deploy/operator/internal/controller`) derives `Phase`/`Ready` from
|
||||
rather than independently guessing. The reconciler itself gained no
|
||||
new credentials and still never touches ClickHouse/Postgres.
|
||||
|
||||
**The deployment-topology gap is closed for the Helm chart**: `deploy/
|
||||
helm/sentry/templates/api.yaml`/`enterprise-api.yaml` are mutually
|
||||
exclusive on `enterprise.enabled`, rendering to the same Service name
|
||||
and port either way, so a Helm-deployed cluster can't accidentally run
|
||||
the wrong binary — the same flag that turns on RBAC/audit/SSO now also
|
||||
chooses the query binary. `docker-compose.yml` still runs plain `api`
|
||||
unconditionally, so this enforcement doesn't yet extend to local/dev.
|
||||
With both storage engines' connection/index-layer mechanisms built and
|
||||
deployment topology enforced at the Helm layer, the largest remaining
|
||||
gaps are ingest's lack of tenant-awareness (undesigned) and unifying the
|
||||
`Tenant` CRD with `-provision-tenant` into one provisioning flow.
|
||||
**The deployment-topology gap is closed for both Helm and
|
||||
docker-compose**: `deploy/helm/sentry/templates/api.yaml`/
|
||||
`enterprise-api.yaml` are mutually exclusive on `enterprise.enabled`,
|
||||
rendering to the same Service name and port either way, so a
|
||||
Helm-deployed cluster can't accidentally run the wrong binary — the same
|
||||
flag that turns on RBAC/audit/SSO now also chooses the query binary.
|
||||
`docker-compose.yml`'s `api`/`enterprise-api` services are the same
|
||||
mutually-exclusive choice via `COMPOSE_PROFILES` (`.env` defaults to
|
||||
plain `api`), sharing a host-port/network-alias trick so `alerting`/
|
||||
`web` need no conditional config either way. With both storage engines'
|
||||
connection/index-layer mechanisms built, deployment topology enforced at
|
||||
both the Helm and docker-compose layers, and the two provisioning
|
||||
mechanisms unified, the largest remaining gap is ingest's lack of
|
||||
tenant-awareness, which is undesigned, not merely unbuilt.
|
||||
|
||||
## Licensing boundary
|
||||
|
||||
|
||||
+51
-4
@@ -482,6 +482,45 @@ against a real daemon in this environment — the `config` rendering above
|
||||
proves the compose file's *shape* is correct, not that containers
|
||||
actually start and route traffic correctly end to end.
|
||||
|
||||
## 11. `Tenant` CRD unified with `-provision-tenant`
|
||||
|
||||
`deploy/helm/sentry/README.md`'s "Trying the two-tenant example" section
|
||||
has the full `helm install` → `-provision-tenant` → `kubectl get
|
||||
tenants` walkthrough. What was actually run in this environment (no
|
||||
live cluster, same limitation as §10):
|
||||
|
||||
```sh
|
||||
cd enterprise && go test ./internal/tenantcrd/... -v
|
||||
# real k8s.io/client-go fake dynamic + typed clientsets, no cluster
|
||||
# needed -- proves Sync() creates/updates the Tenant object and Secret
|
||||
# correctly, is idempotent, never overwrites a pre-existing
|
||||
# spec.displayName, and never rotates a credential across a re-sync.
|
||||
|
||||
cd deploy/operator && go test ./... -v
|
||||
# tenant_controller_test.go rewritten for the new split: proves an
|
||||
# unprovisioned tenant reports Provisioning (not Active -- the
|
||||
# regression test for the pre-unification bug where this controller
|
||||
# claimed Active on its own say-so), that setting
|
||||
# status.clickHouseDatabaseName (simulating what -provision-tenant
|
||||
# writes) flips it to Active, and that un-suspending an
|
||||
# already-provisioned tenant returns straight to Active rather than
|
||||
# being demoted to Provisioning.
|
||||
```
|
||||
|
||||
Helm-side wiring confirmed via `helm template` + parsing the rendered
|
||||
YAML (not eyeballing it): with `tenantOperator.enabled=true`,
|
||||
`enterprise-api` gets its own ServiceAccount/Role/RoleBinding scoped to
|
||||
exactly `tenants`/`tenants/status`/`secrets`, `tenant-operator`'s own
|
||||
ClusterRole no longer grants `secrets` at all, and `TENANT_CRD_NAMESPACE`
|
||||
is only set on `enterprise-api`'s container when `tenantOperator.enabled`
|
||||
is true (absent, and the ServiceAccount/Role absent too, with just
|
||||
`enterprise.enabled=true`). **Not verified**: an actual `-provision-tenant`
|
||||
run against a real cluster with the operator watching -- everything
|
||||
above proves each half's logic and the chart's shape independently, not
|
||||
the full loop (does the operator's watch actually re-trigger a reconcile
|
||||
after `-provision-tenant`'s external status write the way controller-
|
||||
runtime's default predicate is expected to).
|
||||
|
||||
## Known gaps (do not treat this phase as done without reading these)
|
||||
|
||||
Full accounting: `/docs/security/threat-model.md`. Headline items:
|
||||
@@ -494,10 +533,18 @@ Full accounting: `/docs/security/threat-model.md`. Headline items:
|
||||
is set. `docker-compose.yml`'s `api`/`enterprise-api` services are now
|
||||
the same mutually-exclusive choice via `COMPOSE_PROFILES` (§8, §10a),
|
||||
closing the local/dev parity gap this bullet used to name.
|
||||
- The `Tenant` CRD (`deploy/operator`) and `enterprise-api
|
||||
-provision-tenant` are still two independent provisioning mechanisms
|
||||
-- running both for the same tenant ID today takes two separate
|
||||
operator actions, not one.
|
||||
- **The `Tenant` CRD (`deploy/operator`) and `enterprise-api
|
||||
-provision-tenant` are now unified**, in a deliberately lightweight
|
||||
way: `-provision-tenant` stays the sole real actor (ClickHouse +
|
||||
`rbacstore`) and, when `TENANT_CRD_NAMESPACE` is set (the Helm chart
|
||||
does this automatically when `tenantOperator.enabled`), also syncs the
|
||||
real result into the Tenant CRD (`enterprise/internal/tenantcrd`) --
|
||||
see §11 below. Running `-provision-tenant` is still a separate,
|
||||
deliberately manual operator action from `helm install`/`kubectl
|
||||
apply -f tenant.yaml` creating the Tenant object in the first place --
|
||||
that split (declarative request vs. imperative provisioning action)
|
||||
is intentional, not the "two disconnected sources of truth" gap this
|
||||
bullet used to describe.
|
||||
- **Ingest has no tenant concept for either storage engine.** Every
|
||||
record `ingest` produces lands in the one shared ClickHouse database
|
||||
and the one shared Tantivy index no matter what. A newly-provisioned
|
||||
|
||||
Reference in New Issue
Block a user