coffey-labs/cairnobs
1
0
Fork 0
Code Issues Pull Requests 1 Actions Packages Projects Releases Wiki Activity

Close the last tenant-isolation adversarial probe (mid-provisioning tenants)

Browse Source 
Phase 4 task 8's verification plan named four adversarial probes;
three were closed earlier this phase, the fourth (an evaluator tick,
or any other caller, hitting a tenant that exists but hasn't reached
the active+credentialed gate yet -- must be refused, not served) was
still an explicitly-skipped stub in
api/queryapi/tenant_isolation_gap_test.go.

Investigating it found the two storage engines needed genuinely
different treatment:

- ClickHouse (enterprise/internal/chrunner) already had this property
  structurally, for free: Registry is built once at startup from
  rbacstore.ListProvisionedDataSources, which already filters to
  active+credentialed tenants only, so a mid-provisioning tenant is
  simply absent from the connection map. New test
  TestRegistryRefusesMidProvisioningTenant proves this without Docker
  -- an empty DataSource list never dials ClickHouse, so this genuinely
  runs in this environment, unlike every other test in that file.

- Tantivy (search/src/registry.rs's IndexRegistry) was a real, different
  gap, not just an unverified assumption: it opens-or-creates an index
  for any syntactically-valid tenant_id on first request, because it's
  a separate process with no Postgres access and structurally can't
  know which tenants are actually provisioned. A query against a
  mid-provisioning tenant would have silently succeeded with zero
  results from a freshly-created empty index -- "ambient success"
  indistinguishable from "no matching logs," exactly the failure mode
  this item was worried about.

Fixed the Tantivy gap with a new enterprise/internal/searchclient.
TenantChecker interface (backed by a new rbacstore.TenantIsActive,
implemented structurally, no new import edge needed), consulted before
every gRPC call: Client.Search now refuses a non-active tenant before
it ever reaches `search`. Dial's signature gained a required
TenantChecker parameter; enterprise-api's main.go passes its existing
rbacstore.Store (already satisfies the interface). Verified Docker-free
via searchclient's existing real-in-process-gRPC-server test harness
(TestSearchRefusesMidProvisioningTenant, plus
TestSearchPropagatesTenantCheckerError for the fail-closed-on-error
case) -- both genuinely run in this environment, same bar as the rest
of the Tantivy isolation work.

rbacstore.TenantIsActive itself has two new skip-gated live-Postgres
tests (TestTenantIsActive, TestTenantIsActiveNonexistentTenant) --
disclosed as not run against a live database here, same gap as the
rest of this phase's Postgres-backed pieces.

api/queryapi/tenant_isolation_gap_test.go rewritten from a checklist
with one skipped stub to a full accounting of all four now-closed
probes. Docs updated in lockstep: CLAUDE.md, threat-model.md,
phase-4-isolation-design.md (implementation note added after its
original sign-off), phase-4-runbook.md (§9), enterprise/README.md.
This commit is contained in:
jcoffey-dev
2026-08-14 08:07:52 -07:00
parent b8b6a8fd7b
commit 2e698f5623
12 changed files with 345 additions and 51 deletions
Download Patch File Download Diff File Expand all files Collapse all files
enterprise/internal/searchclient/searchclient_test.go
+64 -6
View File
@@ -8,6 +8,7 @@ package searchclient
import (
"context"
"fmt"
"net"
"testing"
@@ -28,7 +29,23 @@ func (f *fakeSearchServer) Search(_ context.Context, req *searchv1.SearchRequest
return &searchv1.SearchResponse{RecordIds: f.recordIDs}, nil
}
func newTestServer(t *testing.T) (*Client, *fakeSearchServer) {
// fakeTenantChecker is an in-memory TenantChecker -- active lists which
// tenant IDs count as active, everything else answers false (not an
// error), matching rbacstore.TenantIsActive's shape (a nonexistent
// tenant is "not active," not a lookup failure).
type fakeTenantChecker struct {
active map[string]bool
err error
}
func (f *fakeTenantChecker) TenantIsActive(_ context.Context, tenantID string) (bool, error) {
if f.err != nil {
return false, f.err
}
return f.active[tenantID], nil
}
func newTestServer(t *testing.T, tenants TenantChecker) (*Client, *fakeSearchServer) {
t.Helper()
lis, err := net.Listen("tcp", "127.0.0.1:0")
if err != nil {
@@ -40,7 +57,7 @@ func newTestServer(t *testing.T) (*Client, *fakeSearchServer) {
go func() { _ = srv.Serve(lis) }()
t.Cleanup(srv.Stop)
client, err := Dial(lis.Addr().String())
client, err := Dial(lis.Addr().String(), tenants)
if err != nil {
t.Fatalf("Dial: %v", err)
}
@@ -49,7 +66,7 @@ func newTestServer(t *testing.T) (*Client, *fakeSearchServer) {
}
func TestSearchForwardsTenantIDFromContext(t *testing.T) {
client, fake := newTestServer(t)
client, fake := newTestServer(t, &fakeTenantChecker{active: map[string]bool{"acme": true}})
ctx := authz.WithIdentity(context.Background(), authz.Identity{TenantID: "acme", Role: authz.RoleViewer})
ids, err := client.Search(ctx, "error", 10)
@@ -68,7 +85,7 @@ func TestSearchForwardsTenantIDFromContext(t *testing.T) {
}
func TestSearchRefusesWithNoIdentity(t *testing.T) {
client, fake := newTestServer(t)
client, fake := newTestServer(t, &fakeTenantChecker{active: map[string]bool{"acme": true}})
if _, err := client.Search(context.Background(), "error", 10); err == nil {
t.Fatal("expected Search to refuse a request with no authenticated identity in context")
@@ -79,7 +96,7 @@ func TestSearchRefusesWithNoIdentity(t *testing.T) {
}
func TestSearchRefusesIdentityWithNoTenant(t *testing.T) {
client, fake := newTestServer(t)
client, fake := newTestServer(t, &fakeTenantChecker{active: map[string]bool{"acme": true}})
// RoleService identities carry no TenantID -- see api/authz.Identity's
// doc comment. /alerting never calls search directly today, but the
// fail-closed behavior must hold regardless of how this arises.
@@ -94,7 +111,7 @@ func TestSearchRefusesIdentityWithNoTenant(t *testing.T) {
}
func TestSearchDifferentTenantsSendDifferentTenantIDs(t *testing.T) {
client, fake := newTestServer(t)
client, fake := newTestServer(t, &fakeTenantChecker{active: map[string]bool{"acme": true, "globex": true}})
ctxA := authz.WithIdentity(context.Background(), authz.Identity{TenantID: "acme", Role: authz.RoleViewer})
if _, err := client.Search(ctxA, "q", 5); err != nil {
@@ -112,3 +129,44 @@ func TestSearchDifferentTenantsSendDifferentTenantIDs(t *testing.T) {
t.Fatalf("TenantId = %q, want globex", fake.lastRequest.TenantId)
}
}
// TestSearchRefusesMidProvisioningTenant is Phase 4 task 8's item 4
// adversarial probe (see /docs/phase-4-isolation-design.md's
// verification plan and api/queryapi/tenant_isolation_gap_test.go),
// closed here without needing Docker or a live Postgres: a tenant that
// exists (an authenticated identity can carry its ID -- e.g. right
// after enterprise-auth -create-tenant / -grant-membership-* but before
// enterprise-api -provision-tenant runs) but isn't active yet must be
// refused, not silently served a fresh empty index. This is exactly the
// gap search/src/registry.rs's IndexRegistry can't close on its own
// (see this package's doc comment) -- proving it here, at the one layer
// that actually has rbacstore access, is what makes the guarantee real.
func TestSearchRefusesMidProvisioningTenant(t *testing.T) {
client, fake := newTestServer(t, &fakeTenantChecker{active: map[string]bool{"acme": true}})
// "pending" is deliberately absent from the active set -- simulates
// a tenant row that exists in rbacstore (provisioning, or even
// active-but-not-yet-credentialed) without needing a real tenants
// table to prove the point: fakeTenantChecker.active only ever
// answers "true" for tenants explicitly marked active, exactly like
// rbacstore.TenantIsActive does for a real 'provisioning' row.
ctx := authz.WithIdentity(context.Background(), authz.Identity{TenantID: "pending", Role: authz.RoleViewer})
if _, err := client.Search(ctx, "error", 10); err == nil {
t.Fatal("expected Search to refuse a tenant that isn't active yet, not silently search an empty index for it")
}
if fake.lastRequest != nil {
t.Fatal("expected the gRPC call to never reach the server for a non-active tenant -- the whole point is refusing before search/src/registry.rs ever gets a chance to open-or-create an index for it")
}
}
func TestSearchPropagatesTenantCheckerError(t *testing.T) {
client, fake := newTestServer(t, &fakeTenantChecker{err: fmt.Errorf("boom")})
ctx := authz.WithIdentity(context.Background(), authz.Identity{TenantID: "acme", Role: authz.RoleViewer})
if _, err := client.Search(ctx, "error", 10); err == nil {
t.Fatal("expected Search to fail closed when the tenant status check itself errors, not treat an error as \"not active but otherwise fine\"")
}
if fake.lastRequest != nil {
t.Fatal("expected the gRPC call to never reach the server when the tenant check errored")
}
}