---
status: active
owner: platform
last_reviewed: 2026-10-01
supersedes: []
---
# Platform LLM — status & architecture (Studio 5-lane)
<!-- wiki:featured -->

Config SSOT: `platform/litellm/config.yaml` → live via `scripts/sync-litellm-config.sh`.
Policy: `platform/shared/llm_policy.py` + `config.py`. Brain: `llm_brain.py`.

## Ownership

| Layer | Path | Role |
|-------|------|------|
| **Venture Studio** | `platform/shared/llm_brain.py`, `llm_policy.py`, `scrubber.py`, `config.py` | task_type→tier, PHI policy, LiteLLM gateway |
| **LiteLLM** | `studio-litellm` (:4000), config `/data/litellm/config.yaml` | `tier/*` → concrete models + in-namespace fallbacks |
| **Astra** | `backend/llm_client.py` | Thin adapter (`product_id="astra-ai-cos"`) |
| **Embeddings** | `platform/shared/embeddings.py` | **Direct Ollama** → pgvector (not via `llm_brain`) |

## Request flow

```
Product API → llm_client / complete_json → llm_brain.complete(task_type)
  ├─ PHI scrub (regex → heuristic → Redis cache → qwen3:0.6b scrubber)
  ├─ policy gate (namespace + thinking invariant) — fail closed
  ├─ enforce + PHI risk → Ollama MODEL_LOCAL_ENFORCE (qwen3:1.7b)
  ├─ LOCAL_ONLY_TASKS → Ollama MODEL_LOCAL
  └─ else → LiteLLM tier/* (5-lane map below)
```

## 5 product lanes (privacy × depth)

| Tier | Namespace | Think | Primary chain |
|------|-----------|-------|---------------|
| `tier/public-fast` | free | OFF | Gemini 3.1 Flash-Lite → OpenRouter free → Vast Qwen3-14B OFF |
| `tier/public-deep` | free | ON→OFF | OpenRouter Llama 3.3 70B free → Vast think ON → Vast OFF |
| `tier/public-batch` | free | OFF | Mistral free → Gemini free → Vast OFF |
| `tier/confidential-fast` | local* | OFF | Vast Qwen3-14B OFF → Ollama qwen3:1.7b (**never free APIs**) |
| `tier/confidential-deep` | local* | ON | Vast think ON → Vast OFF → Ollama (**never free APIs**) |
| `tier/critical` | paid | ON | Gemini 3.8 Flash paid — **fail closed** (no fallback) |
| `tier/local` / `tier/embed` | local | OFF | Oracle Ollama only |

\*Policy namespace `local` = never free/public training APIs; backends are USA Vast + Oracle Ollama.

**Naming:** `tier/veto` is a **legacy alias** for `tier/critical` — `llm_brain._LEGACY_TIER_ALIASES`
still rewrites it for older callers, but the lane's canonical name is `tier/critical`.
Use `tier/critical` in all new code; do not write `tier/veto`. The same applies to the other
retired product-facing names (`tier/fast`, `tier/premium`, `tier/batch`, `tier/legal`,
`tier/sensitive`, `tier/legal-plus`).

**Mental rule:** public vs confidential × fast vs deep; critical = high-stakes paid.

**Invariant:** confidential / critical never fall into `prod/free/*` or OpenRouter free.

## Keys

| Key | Role |
|-----|------|
| `GEMINI_FREE_KEY_1`…`5` | public-fast pool |
| `OPENROUTER_API_KEY` | public-deep |
| `MISTRAL_API_KEY` | public-batch |
| `GEMINI_PAID_API_KEY` | critical (fail-closed) |
| `STUDIO_CLOUD_LLM_URL` + `CLOUD_QWEN14_API_KEY` | Vast Qwen3.5-4B-GPTQ-MTP (served id `qwen3-4b`) |
| `LITELLM_KEY_{INTERACTIVE,WORKBENCH,BACKGROUND}` | prod budget pools |
| `LITELLM_KEY_DEV` | dev/CI/eval isolation |

`STUDIO_LLM_TRAFFIC=dev` → all cloud calls use `LITELLM_KEY_DEV`.

## Validation

```bash
./scripts/validate-llm-brain.sh           # contracts + canaries + tier lint
./scripts/validate-llm-brain.sh --full    # + PHI eval + perf + live tier health
./scripts/validate-phase3a.sh --public
make test-gate
```

| Gate | Script / test |
|------|----------------|
| 5-lane lint | `scripts/validate-llm-tier-config.sh` |
| PHI / sensitivity canaries | `platform/shared/tests/test_llm_policy_canary.py` |
| Live `tier/*` health | `scripts/validate-llm-tier-health.sh` |
| `tier/public-fast` response cache | `platform/shared/response_cache.py` |
| `tier/public-batch` price routing | `platform/shared/batch_price_router.py` |

## Deploy

```bash
# After platform/litellm/config.yaml edits:
./scripts/sync-litellm-config.sh
```