AI Architecture & Security Blueprint

§ 00 — Foreword

Why this document exists

Most "AI security" content is vendor marketing wearing a lab coat, or a checklist lifted from a compliance blog and re-skinned with a stock photo of a padlock. This is neither. It's the reference architecture we actually reach for when we build AI systems for organizations that can't afford to get this wrong — regulated finance, defense-adjacent contractors, anyone whose general counsel reads the trade press.

Every figure below is sourced to a primary document — a standards body, a vendor's own engineering blog, an SEC release, a peer-reviewed paper — not an aggregator's listicle. Where something is still unproven, we say so. Agent-to-agent payments, for instance, are real and moving fast, but they are not yet a production default, and we've labeled them accordingly.

§ 01 — Diagnosis

The barrier was never the model

For two years, the industry conversation about enterprise AI adoption centered on capability — was the model smart enough, fast enough, cheap enough. That conversation is over. Capability stopped being the constraint sometime in 2025. What replaced it is trust, and the data is specific about where that trust breaks down.

McKinsey's State of AI Trust in 2026 found that nearly two-thirds of respondents cite security and risk concerns as the top barrier to fully scaling agentic AI — ahead of regulatory uncertainty, ahead of talent, ahead of cost.^[1] Within that group, 74% flagged inaccuracy and 72% flagged cybersecurity as highly relevant risks to their organization.^[1] Gartner's newsroom named agentic AI's expanding attack surface the #1 cybersecurity trend for 2026, pointing specifically to "unmanaged AI agent proliferation, unsecured code, and potential regulatory compliance violations."^[2]

Gartner's forward numbers are worth sitting with, and worth reading as forecasts, not measurements: by 2028, a quarter of enterprise generative-AI applications are projected to suffer five or more minor security incidents a year, up from 9% in 2025; by 2029, 15% are projected to have at least one major incident annually, up from 3% in 2025.^[3] A separate Gartner market guide on "guardian agents" puts a finer point on where those incidents originate: through 2028, at least 80% of unauthorized AI agent transactions are projected to trace back to internal policy violations around information oversharing — not external attackers.^[4]

That last statistic is the one that should reshape how you think about this problem. The threat model for agentic AI is mostly internal, mostly about scope and permissions, and mostly solvable with architecture rather than a new SKU. The rest of this document is that architecture.

§ 02 — Infrastructure

Build the fabric for east-west, not north-south

Why this is on your desk, not just your engineers': get the network topology wrong and everything you build on top of it — latency, cost, the ability to scale without a re-architecture — inherits the mistake. This section is what to check before you approve a vendor's diagram, not a tutorial.

Every legacy network was designed around a simple assumption: most traffic goes to the internet. Core-aggregation-access topologies optimized for that north-south flow. AI training and inference invert it completely — the dominant traffic is GPU-to-GPU gradient sync, GPU-to-storage checkpointing, server-to-server east-west flow that barely touches the internet at all. A three-tier legacy design can't equalize latency for that pattern, which is exactly why it's been displaced by Clos / spine-leaf fabric across every serious AI buildout.^[5]

The mechanics: leaf switches connect directly to compute nodes; spine switches connect to every leaf, so any two nodes are always the same number of hops apart. IBM's HPC reference architecture is blunt about why this specific topology won: "the fat-tree topology is the most common one for InfiniBand … [it] provides a fully non-blocking network if the uplinks have the same throughput as the links to the nodes."^[5] NVIDIA's DGX SuperPOD reference architecture implements exactly this — a three-tier fat-tree over Quantum-2 InfiniBand at 400 Gb/s per port, sized for full bisection bandwidth.^[6]

Inside a rack, NVLink does the tightest-coupled work: NVIDIA's GB200 NVL72 puts 72 GPUs in a single NVLink domain at 900 GB/s, and NVIDIA's own documentation labels this GPU compute fabric "east-west," reserving "north-south" for the converged fabric that talks to the rest of the network.^[7] Get this topology wrong and every other layer in this document inherits the latency problem — orchestration, retrieval, and inference all sit on top of it.

§ 03 — Orchestration

Orchestration you can put in front of an auditor

Why this is on your desk, not just your engineers': this is the layer that determines whether "the AI made a mistake" is an acceptable answer to your board, or a liability. An agent that can call tools and an agent you can audit are two different engineering problems, and most agent frameworks only solve the first one. The stack below solves both by separating four concerns that are usually tangled together: reasoning, durability, access control, and output shape.

LangGraph handles the reasoning graph — a low-level orchestration layer built for long-running, stateful agents, with durable execution, streaming, and human-in-the-loop checkpoints baked in. Its distinguishing feature is the ability to mix deterministic steps with LLM-driven agentic steps inside a single graph, rather than forcing everything through a model call.^[8] It's in production at Klarna, Replit, and Elastic.^[8]

Temporal sits underneath as the durable execution engine — the part that survives a crash mid-workflow. Temporal now ships an official LangGraph plugin, and the framing its own engineers use is exactly right: LangGraph is the brain, Temporal is the muscle. Tool calls become retryable activities; the reasoning loop becomes a workflow that survives a process restart without losing state.^[9]

Tool access is enforced at an MCP gateway, not by convention. Red Hat's OpenShift AI 3 ships MCP support with role-based access control and OAuth, and its gateway enforces policy and RBAC rules on every call rather than trusting the agent to self-limit.^[10] The field is already moving past static RBAC toward context- and attribute-based access control — CBAC/ABAC — plus tool-level allow-lists, a shift documented by Aembit, Cerbos, and a recent arXiv survey of MCP security patterns.^[11] Build for that direction now; retrofitting access control after an incident is a much worse project.

Finally, every model output that a downstream system will act on is passed through XGrammar, a constrained-decoding engine out of CMU, NVIDIA, SJTU, and UC Berkeley.^[12] It pairs a pushdown automaton with an adaptive token-mask cache to guarantee structurally valid JSON, regex, or grammar output with near-zero overhead — up to 14× faster than naive constrained decoding on JSON schemas, 80× faster on full context-free grammars.^[12] It's now the default structured-generation backend in vLLM, SGLang, TensorRT-LLM, and MLC-LLM. In practice: your agent cannot emit a malformed instruction to a system that executes trades, writes to a database, or files a ticket. It's not a possibility you handle with error-checking downstream — it's structurally excluded upstream.

§ 04 — Deployment model

Keep inference inside your own perimeter

"Sovereign AI" gets used two ways, and both matter here. At the national-policy level — NVIDIA's own framing, echoed in EU, UAE, and Australian frameworks — it means treating where AI processing happens like a data-residency question, not just where data is stored.^[13] At the architecture level, it's narrower and more actionable: build, deploy, and govern AI with real control over your data, infrastructure, and models, instead of routing every inference call through a third party's API and hoping their terms of service age well.

The drivers are concrete, not ideological: data residency obligations under GDPR, sector rules in BFSI and healthcare (HIPAA), defense contract requirements, and the simple desire to keep proprietary code and data from ever crossing a boundary you don't control. The serving stack that makes this practical today is vLLM and TensorRT-LLM for inference, with models running inside your VPC and retrieval enforced through the same RBAC layer covered in §03 — not a separate, weaker permission system bolted on for the AI use case.

§ 05 — Case reference

Proof at scale: NCHC Nano4

Taiwan's National Center for High-Performance Computing, working with ASUS, built a dual-architecture cluster combining NVIDIA HGX H200 (locally named "晶創26," Nano4) with a GB200 NVL72 system. Nano4 debuted provisionally at #29 on the November 2025 TOP500 list at SC25. On the official June 2026 TOP500 list, after formal entry into operation on June 1, 2026, it settled at #33 — 81.55–81.6 PFLOPS Rmax across 256,960 cores, drawing 2.214 MW.^[14]

Two operational numbers matter more than the ranking. First, PUE 1.18, achieved through direct liquid cooling — a power-usage-effectiveness figure that puts it near the efficiency ceiling for air-adjacent facilities. Second, ASUS's own deployment automation reduced the buildout timeline from three weeks to three days, a figure ASUS has confirmed verbatim in its own press materials.^[14] Both numbers are the direct product of the fabric and automation discipline described in §02 — they aren't a separate achievement, they're what that architecture looks like at supercomputer scale.

§ 06 — Code intelligence

Code agents that actually learn your codebase

Worth being precise here, because the product landscape gets misrepresented constantly, including in earlier drafts of this document. Sourcegraph Cody is a retrieval system, not a fine-tuned one. Sourcegraph's own engineering blog is explicit: "Today, we use RAG to power Cody's responses … for code retrieval specifically, we find RAG to be the more appropriate tool for the job."^[15] Cody combines BM25-style code search with local IDE context — no per-customer fine-tuning is documented anywhere in Sourcegraph's own materials. Sourcegraph discontinued Cody Free and Pro in July 2025 and now points users toward Amp, spun off as a separate company that December; Cody Enterprise remains, and remains retrieval-based.

If you specifically want fine-tuning on your own repositories, two products actually do it. Tabnine's "Trainbox" retrains a universal model on a customer's selected private repos.^[16] Refact.ai fine-tunes on internal codebases directly and reports the share of AI-written code in its customers rising from 25% to 45% after fine-tuning.^[16]

The more interesting data point is how cheap per-repo specialization has become. Ai2's SERA — the first release in its "Open Coding Agents" line, published January 27, 2026 — resolves 54.2% of SWE-bench Verified with SERA-32B, trained on a cluster of just two NVIDIA Hopper GPUs using a technique Ai2 calls Soft-Verified Generation.^[17] Total cost for data generation and training: $2,000, or 40 GPU-days. Ai2's own number for adapting it to a single unfamiliar repository is roughly 8,000 trajectories — about $1,300.^[17] That figure is the one to remember: specializing a coding agent to your specific codebase is no longer a research-lab budget line, it's closer to a line item.

§ 07 — Model selection

The 2026 open-weight roster

The thesis behind sovereign deployment only holds if the open-weight models are actually competitive, and as of mid-2026 they are — matching or beating closed APIs on many coding and reasoning benchmarks while running entirely inside infrastructure you control. License terms vary more than capability does, so read the second column carefully.

If clean commercial licensing is a hard requirement — which it usually is, the moment legal gets involved — Qwen3, Gemma 3, and Mistral Small 3.x are the Apache-2.0 picks. Llama 4 is capable but carries Meta's custom license and its 700-million-monthly-active-user cap, which matters more than people expect once you're building something that might actually succeed.

§ 08 — Compliance

Recordkeeping that survives an audit

For regulated financial firms, the controlling rule for electronic records is SEC Rule 17a-4 (17 CFR § 240.17a-4) — the broker-dealer recordkeeping and retention rule, companion to Rule 17a-3, which governs what records must be created in the first place. Historically, it required electronic records to be kept exclusively in non-rewriteable, non-erasable WORM format. On October 12, 2022, the SEC adopted amendments, effective January 3, 2023, that kept WORM as an option and added an audit-trail alternative — a system that permits recreation of an original record if it's later modified or deleted.^[18] Your electronic recordkeeping system needs to satisfy one of the two, not necessarily WORM specifically.

Retention periods, per §240.17a-4 and FINRA Rule 4511: daily blotters, general ledgers, and customer account ledgers — 6 years, with the first two years readily accessible. Most other records, including written agreements, bank records, bills, and communications — 3 years, first two accessible. Customer account records — 6 years after account closure. Articles of incorporation and minute books — life of the firm plus 3 years.

§ 09 — Emerging

Where agent-to-agent payments are heading

Card rails were never built for what agents need. Stripe's standard US rate — 2.9% + $0.30 — is dominated by that fixed $0.30 the moment payments get small: a $25 transaction carries an effective rate near 4.1%, and anything under roughly $10 can see an effective rate double the headline percentage.^[19] Sub-cent, agent-initiated micro-transactions are simply uneconomical on that rail, and that gap is real, not theoretical.

x402 is the leading attempt to close it — an open protocol Coinbase launched in May 2025, built on the long-dormant HTTP 402 "Payment Required" status code, settling instantly per-request in USDC.^[20] A V2 shipped within six months, and the x402 Foundation, co-founded with Cloudflare, now governs the spec. On high-throughput chains the settlement economics look genuinely different from card rails — Solana's own documentation cites roughly $0.00025 per transaction with about 400ms finality.^[21] Ecosystem signals are accumulating fast: Amazon Bedrock AgentCore Payments entered preview on May 7, 2026 with native x402 support; Google's Agent Payments Protocol (AP2) incorporates x402 as well, with Cloudflare, Google, Vercel, and Coinbase all backing the standard.^[22]

§ 10 — Closing

What to do with this

Everything above is the reference architecture, not a product pitch — you can take the fabric design, the LangGraph/Temporal/MCP/XGrammar stack, the model roster, and the compliance checklist and build all of it yourselves. Plenty of teams do exactly that, and should.

Here's the honest math for the teams that don't have a platform engineer to spare. It's a working estimate, not a sourced statistic — treat it as a napkin, not a citation.

The gap isn't really the dollars — it's the six to ten weeks your best engineer spends on infrastructure instead of the product you're actually trying to ship. That's the trade the note below is for.