How does tenant isolation apply to shared foundation models?

The foundation model (GPT-5, Claude 4, Gemini 3) is shared across all tenants of the platform. The isolation applies to the data that flows through the model, not to the model itself. Each tenant's prompt, retrieval context, tool call, and response are scoped to the tenant. The model has no tenant awareness; the platform enforces the tenant boundary at the application and gateway layers.

Can we use a separate model deployment per tenant?

Some enterprise contracts require per-tenant model deployment (dedicated instances on AWS Bedrock, Azure OpenAI, or GCP Vertex). Per-tenant deployment adds a second layer of isolation and simplifies the compliance story, but multiplies the cost. Most multi-tenant SaaS starts with a shared deployment and moves specific tenants to dedicated deployments based on contract requirements.

What happens if the model returns content that leaks the system prompt?

System prompt leakage is a common attack pattern. The gateway's response classifier can catch known system prompt fragments in the response. The application's system prompt should not contain tenant-specific information; tenant-specific configuration goes in the prompt context rather than the system prompt.

How do we handle tenants that share integrations with the same downstream vendor?

The tool call isolation applies at the tenant credential layer. Each tenant configures its own credentials to the downstream vendor. Two tenants of the platform who both integrate with the same vendor use separate credentials. The gateway selects the correct credentials based on the tenant identity of the request.

Do we need to provide tenant-specific model fine-tuning?

Model fine-tuning per tenant is expensive and rarely necessary for isolation. Most multi-tenant deployments use a shared model with tenant-scoped context. Fine-tuning is warranted when tenant-specific vocabulary or task requirements benefit from the customization, not for isolation reasons.

AI Tenant Isolation: How Multi-Tenant SaaS Enforces Per-Customer Boundaries on LLM Traffic

Multi-tenant SaaS applications enforce their existing boundary in three places: database rows carry a tenant_id, queries apply that ID, and object storage uses tenant-prefixed paths. LLM features add four paths carrying tenant context outside that database filter: the prompt, retrieval context, an agent tool call, and the response. Each path can place tenant A's data in tenant B's context without a database join. A multi-tenant application needs controls at each path.

I want to walk through the four isolation domains, the enforcement patterns at the AI gateway, and the audit records that demonstrate the isolation held across the audit period.

TL;DR

Tenant isolation needs an application-enforced scope on prompts, retrieval, tool calls, and responses. DeepInspect can enforce and record tenant-scoped policy for routed HTTP LLM requests. The application and downstream services retain responsibility for retrieval filters, credentials, and local tool execution.

The four isolation domains

Prompt isolation covers the request body the application sends to the LLM provider. The prompt contains data from the tenant's context: their customer records, their conversation history, their business rules, their system-prompt-carrying tenant configuration. The isolation obligation is that the prompt for a request executing under tenant A's identity contains no data from tenant B's context.

Retrieval isolation covers the RAG pipeline. The vector store holds embeddings from documents each tenant contributed. A retrieval query executed for tenant A has to return only vectors from tenant A's documents. The vector store's query path has to enforce the tenant filter the same way the database enforces its tenant filter.

Tool call isolation covers the agent's outbound calls to systems that hold tenant data. When an agent runs under tenant A's identity and calls the tenant's CRM, the CRM call has to authenticate as tenant A and access only tenant A's records. The tool call has to preserve the tenant identity across the boundary.

Response isolation covers the LLM output. The model's response has to be scoped to the tenant. Model providers do not have a tenant model; the model returns whatever text it produces. The application (or the gateway) has to ensure the response does not contain data the model surfaced from another tenant's context inadvertently.

The prompt isolation enforcement pattern

Prompt isolation starts at the application. When the application constructs the prompt, the application selects the context that belongs to the tenant. Selection is a database query, a cache lookup, or a session-scoped variable, and each has to filter by tenant.

The gateway adds a second control. The gateway resolves the tenant identity from the request (from the enterprise SSO token, from the API key, or from a tenant header). The gateway's classifier scans the prompt for cross-tenant markers, such as a customer identifier prefix, account name, or document identifier assigned to another tenant.

The gateway's classifier can be pattern-based (regex on known tenant identifier prefixes) or embedding-based (semantic similarity to tenant-bound content). Pattern-based classifiers are deterministic and cheap; embedding-based classifiers cover more content but require the tenant's authoritative embeddings.

When the classifier flags a cross-tenant pattern, the gateway denies the request and logs the deny event. The application's next request has to originate from the correct tenant scope. The deny event is a signal to the SOC that the application has an isolation defect.

The retrieval isolation enforcement pattern

RAG pipelines add a metadata field to each vector: the tenant identifier. The query engine has to include the tenant filter in every query. The filter is not optional and is not the application's choice at query time.

The gateway enforces the filter at the retrieval boundary. When the application calls the vector store through the gateway, the gateway inspects the query and rejects any query that does not include the tenant filter. The gateway can also add the filter to queries that omit it, using the tenant identity from the request context.

The vector store access model has to align with the application control. When its API accepts a query without the tenant filter, the application path becomes the sole barrier. A store that enforces tenant filtering independently gives an auditor a second control to test.

The tool call isolation enforcement pattern

Agent tool calls to downstream systems (CRM, database, ticketing system, file store) have to authenticate as the tenant. Two patterns satisfy the requirement.

Tenant-scoped credentials. The tenant configures per-tenant credentials for each downstream system. The agent uses the tenant's credentials to call the system. The system authenticates the credentials and returns only the tenant's data.

Delegation. The application authenticates to the downstream system as the platform and asserts the tenant identity through a delegation token. The downstream system's authorization applies the tenant scope to the query.

Tool-call isolation belongs in the application and downstream authorization layer. DeepInspect's HTTP AI-traffic boundary does not cover local tool execution, credential retrieval, or downstream CRM and database calls. The application should keep raw credentials out of model context and issue tenant-scoped credentials or delegation tokens to the downstream service.

The response isolation enforcement pattern

Response isolation runs against the model output. The gateway inspects the response for another tenant's context, internal platform information, or system-prompt fragments that the model surfaced.

The response classifier can be rule-based (deny responses containing known tenant identifier patterns from other tenants) or embedding-based (deny responses semantically close to content the tenant does not own).

The response classifier is more difficult to implement well than the prompt classifier because the response is free-form generated text. False positives on legitimate responses reduce the feature's utility. False negatives on cross-tenant content are the isolation failure the classifier is supposed to catch.

High-sensitivity tenants often need a tighter response policy and human-review escalation. A regulated healthcare deployment can use a lower detection threshold than a product-feedback assistant that handles no protected records.

The audit records that demonstrate isolation

The audit records prove the isolation held over the audit period. The records have to answer three questions.

Which tenant executed each request? The gateway records the tenant identity resolved at request time.

Did the request contain cross-tenant content? The gateway records the classifier verdict on each request.

Did the response contain cross-tenant content? The gateway records the response classifier verdict on each request.

The record series has to be queryable per tenant. The tenant's own compliance officer can request an audit of the tenant's requests, and the platform has to produce the record excerpt for the tenant without exposing other tenants' records. The tenant-scoped audit access is itself an isolation obligation.

The interaction with SOC 2 and ISO 42001

SOC 2 Common Criteria CC6 covers logical access controls. The AICPA Trust Services Criteria provides the source material an audit team maps to its control descriptions. The auditor tests whether the platform maintains tenant isolation across the AI request path. Tenant identity resolution, classifier verdicts, and audit records can supply evidence.

ISO 42001 Annex A.8.4 (responsible use logging) and A.9.4 (intended use monitoring) apply to tenant-isolation records. NIST's AI Risk Management Framework also gives teams a public reference for documenting and testing risk controls. An auditor can sample records and test whether the isolation control held.

Confidentiality and privacy trust services criteria (when the SOC 2 engagement includes them) add specific isolation obligations. The platform's contractual commitments to specific tenants (isolation SLAs in enterprise agreements) also add obligations the audit has to verify.

The interaction with EU AI Act Article 26

Article 26 assigns the deployer specific obligations. In a multi-tenant SaaS setup, each tenant is a separate deployer under the AI Act. The platform provides the AI system, and the tenant deploys the AI system within its own environment.

The platform's tenant isolation architecture is what allows each tenant to satisfy its own Article 26 obligation. The tenant needs to demonstrate that its AI system's operation is isolated from other tenants. The platform's audit records support the tenant's demonstration.

Enterprise contracts increasingly include specific Article 26 provisions. The tenant asks the platform to warrant tenant isolation and to produce audit records the tenant can share with its own auditor. The platform's contract terms and the audit evidence have to align.

DeepInspect

DeepInspect intercepts HTTP traffic between authenticated users or agents and LLM APIs. It can resolve a tenant identity carried in that request, apply a tenant-scoped policy to the prompt, and record the policy decision with the destination model and outcome. AI gateway architecture explains that HTTP request boundary in more detail.

The record supports a tenant-scoped audit trail for routed LLM requests. Retrieval filters, tool credentials, vector-store queries, and downstream system permissions remain application and service controls. AI audit-log immutability and SOC 2 controls for AI describe the evidence and access-control layers that complete the design.

Book a technical deep dive at deepinspect.ai.