What Is an AI Gateway? What It Governs, What It Doesn't, and Why Employee AI Still Needs the Endpoint

Answer first: An AI gateway (also called an LLM gateway) is a proxy that sits between the applications your company builds and the model providers they call, adding routing, failover, caching, rate limits, spend controls, and logging to API traffic. It governs the AI your developers ship. It does not govern the AI your employees use in ChatGPT, Claude, Copilot, Cursor, or a browser tab, because that traffic never passes through it. dope.security governs the employee side from the endpoint, where every AI app actually sends its data.

"AI gateway" is one of the most confusing terms in security right now, and vendors are not helping. Cloudflare sells one. Kong sells one. Portkey and LiteLLM are built around the idea. Netskope and Zscaler both announced products with "gateway" or "broker" in the name this year. A CISO hears "we have an AI gateway" in a board update and assumes the shadow AI problem is handled. It usually is not. If you are starting from the broader question, our buyer's guide to choosing an AI governance platform maps the five architectures; this post zooms in on the one that gets mislabeled most.

What an AI gateway actually does

Strip the marketing and an AI gateway is an API proxy with model-aware features. Your application calls one endpoint; the gateway forwards the request to OpenAI, Anthropic, Google, Bedrock, or a self-hosted model, and handles the operational chores around that call. Across the major products, the feature set is consistent:

• Routing and failover. Send a request to a cheaper model, fall back to another provider when one is down, or run A/B tests across models from a single config.
• Caching. Return a stored answer for a repeated or semantically similar prompt instead of paying for a new completion.
• Rate limiting and spend controls. Cap tokens or dollars per team, per key, or per model so one runaway job does not eat the budget.
• Observability. Log every prompt and response, with latency, cost, and token counts, for debugging and chargeback.
• Guardrails. Some gateways add PII redaction, prompt-injection screening, or content filters to the request path.

Cloudflare AI Gateway proxies LLM API calls through Cloudflare's edge and is popular because its free tier gives teams basic visibility fast. Kong AI Gateway extends the Kong API gateway with AI plugins, which suits teams already running Kong. Portkey positions itself as a governance control plane with guardrails and prompt management. LiteLLM is the widely adopted open-source option. All of them are excellent at what they do, and what they do is manage API calls that your own software makes.

What an AI gateway does not see

Here is the test. Open ChatGPT in Chrome, paste a customer list, and hit enter. Did that request go through your AI gateway? No. ChatGPT's web app talks to OpenAI directly from the browser. Open Claude Desktop and attach a contract. Same answer. Run Claude Code in a terminal against your repo. Same answer. Install a browser extension that summarizes pages with Gemini. Same answer.

An AI gateway only sees traffic that is explicitly pointed at it, which means traffic from code your engineers wrote and configured. Employee AI use is the opposite: hundreds of SaaS apps, desktop clients, IDE plugins, and extensions, each with its own hard-coded endpoint. None of them will ever be configured to route through your gateway, and you cannot make them. That is the entire shadow AI problem, and a gateway is structurally on the wrong side of it. Our guide to detecting shadow AI in desktop apps, IDEs, and CLIs shows how much of the traffic lives outside the browser, let alone outside a developer proxy.

Two distinct problems, then. The gateway problem: govern the model calls our products make. The governance problem: see and control the AI our people use. A tool built for the first does not solve the second, no matter what it is called.

Why the SSE vendors are blurring the line

This is where naming gets slippery. In 2026 the legacy proxy vendors each launched an AI product family and borrowed the gateway vocabulary.

• Netskope rebranded its AI products as Netskope Skylight on September 15, 2026, with an AI Gateway and an Agentic Broker in the lineup. Skylight Agent Action Control was announced for end of quarter rather than shipping at announcement, and the tier it lands in was not disclosed. The employee-facing pieces still run through the NewEdge cloud proxy, which Netskope's own SLA rates at 50 ms for decrypted traffic.
• Zscaler announced AI Broker for agent and MCP traffic, Endpoint AI Security, and AI Access Graph at Zenith Live on June 9, 2026, with GA and pricing undisclosed. Prompt DLP on Zscaler already requires the Data Protection add-on, and AI Guard and AI Scanning are separately licensed. More products to license and integrate, not a single control point.
• Palo Alto Networks announced a native integration between Prisma AIRS AI Gateway and its SSE, not yet shipped, while employee AI controls remain AI Access Security stacked on CASB and Enterprise DLP SKUs.
• Cloudflare is the clearest case: AI Gateway is a developer proxy, full stop. Employee governance lives in Cloudflare One's Gateway, where AI Prompt Protection covers four apps and shadow-MCP detection, launched August 14, 2026, is an experimental selector that only works on traffic already under TLS inspection.

The pattern is the same everywhere: a developer gateway for your apps, a separate employee-governance SKU bound to the cloud proxy, and a slide that puts them under one umbrella. Our breakdown of why AI security on Zscaler is a stack of add-on SKUs walks the licensing in detail. None of this is "they have no AI." It is "the employee side is a bolt-on to the proxy, and the proxy only sees what it decrypts."

Where employee AI governance has to run

If the data leaves from the device, the control has to be on the device. That is not a slogan; it is the only place that sees a paste into ChatGPT Desktop, a file attached in Claude, a repo pushed from a terminal, and a browser session, all with the same visibility. dope.security runs a single agent on Mac and Windows that inspects SSL locally and applies three layers of AI governance:

• Discover. AI Analytics shows every AI app on every endpoint, who used it, how much data moved, and whether the account was personal or enterprise-licensed. No proxy logs to grep, no survey.
• Control access. SWG policy allows, warns, or blocks AI categories, and Cloud Application Control restricts ChatGPT, Claude, Google, and Microsoft 365 to your corporate tenants. Allow corporate ChatGPT, block the personal login on the same domain. DNS cannot do that; a developer gateway cannot either.
• Inspect the data. Dopamine DLP reads prompts and attachments on the device and classifies them by meaning, so a customer list or a block of source code is blocked before it leaves, with a plain-language summary of why. Coverage spans ChatGPT, Claude, Perplexity, Abacus, and Copilot.

All of it is native to the platform. There is no AI SKU to add, no second console, and no cloud detour, which is why the agent stays under 100 MB of RAM and runs up to 4x faster than legacy proxy SWGs.

AI gateway vs endpoint AI governance: head to head

Two tools, two jobs. Here is the comparison that should sit in every AI governance plan.

An AI gateway (Cloudflare, Kong, Portkey, LiteLLM, or a vendor "AI broker")

Governs model calls made by applications you build and configure. Strong on routing, failover, caching, token budgets, and per-key logging. Sees nothing from employee SaaS AI, desktop AI apps, IDE plugins, or browser extensions, because those endpoints are hard-coded by their vendors. When an SSE vendor extends the idea to employees, the enforcement point becomes the cloud proxy, with the latency, outage exposure, and add-on licensing that implies.

dope.security endpoint AI governance

Governs the AI your employees use, from the device. Discovers every AI app, attributes usage per user and account type, enforces tenant-only access with Cloud Application Control, and classifies prompts and uploads with Dopamine DLP, all in one on-device agent with no proxy detour. It does not route your production app's API calls; that remains a job for a developer gateway, and the two run side by side without conflict.

The paired view

• Traffic covered: a gateway covers configured app-to-model calls; dope.security covers every process on the endpoint.
• Shadow AI: a gateway is blind to it by design; dope.security's AI Analytics surfaces it instantly.
• Personal vs corporate accounts: a gateway has no concept of this; Cloud Application Control enforces it per tenant.
• Enforcement point: gateway in the cloud or a container; dope.security on the device, so policy holds on any network.
• Licensing: SSE vendor AI products are new SKUs with undisclosed pricing; dope.security includes AI governance in the base platform.

A quick way to tell which one a vendor is selling

Ask three questions on the call. Does the product see a prompt typed into ChatGPT's web app by a sales rep who has never heard of your gateway? If the answer involves "once you configure the application to route through us," it is a developer gateway. Does it distinguish a personal ChatGPT login from your enterprise tenant on the same domain? That requires reading a header inside decrypted TLS on the user's device, which a gateway never touches. Does it see Claude Code or Cursor sending a file from a terminal? If the answer is a browser extension, you have found the boundary. A tool that passes all three is doing endpoint governance, whatever it is called. Our guide to MCP server security applies the same test to agent traffic, which is where the gap is widening fastest.

How the two fit together in a real stack

A healthy 2026 AI program usually has both. Engineering runs an AI gateway in front of the models their products call, with spend caps and logging that finance can read. Security runs endpoint governance across the fleet so that the far larger share of AI usage, the part that happens in SaaS apps and desktop clients, is visible and controlled. The gateway's logs and the endpoint's AI Analytics answer different questions, and a CISO needs both answers. What a CISO does not need is a cloud-proxy vendor charging for a third thing in between.

A mid-market energy company's AI committee asked its DNS-based vendor one question it could not answer: who is using which AI tool with what data. The team ended up replacing Cisco Umbrella because the answer required endpoint visibility, not a better dashboard. The same logic applies to a gateway: it can only report on traffic it was told about.

The bottom line

An AI gateway is a developer tool for the AI you build. Employee AI governance is a security control for the AI your people use, and it only works where the data leaves: on the device. Buy the gateway for engineering if you need it. Put dope.security on the endpoints so the rest of the company's AI use is discovered, attributed, and controlled from one console. Start a free trial and run AI Analytics for a week; the first report usually settles the debate.

Frequently Asked Questions

What is the difference between an AI gateway and an API gateway?

An API gateway proxies any HTTP API with auth, rate limiting, and routing. An AI gateway is an API gateway specialized for model providers: it understands tokens, models, and prompts, so it can do model failover, semantic caching, token budgets, and prompt logging. Kong AI Gateway is literally the Kong API gateway with AI plugins. Neither one sees employee AI use in SaaS apps or desktop clients; dope.security covers that from the endpoint.

Does an AI gateway stop employees from pasting sensitive data into ChatGPT?

No. ChatGPT's web app and desktop app send data straight to OpenAI and never pass through your gateway. Stopping that paste requires a control on the device. dope.security's Dopamine DLP reads the prompt locally, classifies it by meaning, and blocks it before it leaves, across ChatGPT, Claude, Perplexity, Abacus, and Copilot.

Is Cloudflare AI Gateway an AI governance tool?

Cloudflare AI Gateway is a developer proxy for LLM API calls, with routing, caching, and spend controls. It is not an employee governance tool. Cloudflare's employee-side AI controls live in Cloudflare One Gateway, where AI Prompt Protection covers four apps and shadow-MCP detection is experimental and depends on TLS inspection. dope.security enforces AI governance on the device across every app.

Do I need an AI gateway if I use dope.security?

Only if your engineering team builds applications that call model APIs and wants routing, failover, or token budgets for those calls. dope.security governs employee AI use from the endpoint and does not replace a developer gateway. The two run side by side: the gateway for your products, dope.security for your people.

Is Netskope Skylight or Zscaler AI Broker an AI gateway?

Both borrow the vocabulary. Netskope Skylight, announced September 15, 2026, includes an AI Gateway and an Agentic Broker; Zscaler announced AI Broker on June 9, 2026 with GA and pricing undisclosed. In both cases employee enforcement runs through the vendor's cloud proxy and is licensed as additional SKUs. dope.security ships AI governance natively in a single on-device agent.

Can an AI gateway see MCP servers employees connect to AI tools?

Only if the MCP traffic is deliberately routed through it, which employee tools do not do. MCP connections open from desktop apps and CLIs on the endpoint. dope.security sees them at the device level, which is why on-device architecture is the practical answer for shadow MCP as well as shadow AI.