Shadow AI Monitoring: What to Track and How to Report It
.jpeg)
Quick answer: Shadow AI monitoring is the ongoing practice of measuring which AI tools your employees use, who uses them, on what kind of account, and what data leaves with the prompt. Track five things: total AI requests, active AI users, distinct AI apps detected, applications per user, and personal versus enterprise account usage. dope.security is the strongest fit because the agent runs on the device with on-device TLS inspection, so it sees browser tabs, desktop apps and CLI tools alike, and its AI Usage Analytics dashboard turns that telemetry into a rolling 7-day view with a branded PDF export for your CISO.
New here? Start with our step-by-step guide to detecting shadow AI or our conceptual explainer on shadow AI detection.
Discovery is a project. Monitoring is a discipline.
Most teams run a shadow AI discovery exercise once, produce a spreadsheet, feel briefly alarmed, and then let the spreadsheet rot. Three months later the tool mix has changed completely, because it changes faster than any other category of software in your environment.
The average company uses 10x more AI tools than IT approved. That ratio isn't a one-time finding. It's a moving number, and the only useful way to hold it is a repeating measurement with a fixed cadence and a fixed set of metrics.
Shadow AI monitoring is what you do after discovery. It answers a different question. Discovery asks "what's out there." Monitoring asks "what changed, does it matter, and do I need to act."
What shadow AI monitoring actually measures
Four data points describe any instance of AI usage, and a monitoring program that captures fewer than four leaves a gap someone will eventually find.
The tool. Which AI application handled the request. ChatGPT, Claude, Gemini, Copilot, Perplexity and Abacus are the named tools dope.security covers today, and the long tail behind them grows every month.
The account. Whether the user signed in with a corporate tenant or a personal one. This is the single most important field in the entire dataset, and most monitoring approaches never capture it. Cloud Application Control reads the tenant header inside decrypted TLS, which is exactly why DNS-layer tools can't produce this field.
The person. Which user, on which device. Aggregate counts tell you the scale of the problem. Per-user counts tell you where to start.
The data. What went into the prompt or the upload. Dopamine DLP intercepts file uploads and AI prompts and classifies them with LLMs rather than regex, detecting PII, PCI, PHI and IP before the data reaches a model.
Drop any one of those and your monthly report gets vague in a way executives notice.
The metrics that matter, and when to act on them
AI Usage Analytics in dope.console is built from the agent's AI-traffic telemetry across every endpoint. Here's how to read the numbers it surfaces.
| Metric | What it tells you | When to act |
|---|---|---|
| Total AI Requests (rolling 7-day) | Overall adoption velocity across the fleet | A sustained 3-week climb means your sanctioned tooling isn't keeping pace with demand |
| Active AI Users | How broad adoption is, not just how loud | When this crosses a majority of the fleet, AI becomes a governance question, not an exception process |
| Distinct AI Apps Detected | Tool sprawl | Any new app appearing in the list should get a named owner within the week |
| Top AI Applications (by transactions) | Where the volume concentrates | The top 3 apps are your sanctioning shortlist. License them or control them. |
| Top AI Applications (by number of users) | Where adoption is broad rather than deep | A tool with many users and few transactions each is a candidate for a standard, approved alternative |
| Top AI Users (by transaction volume) | Your power users | These people are not the problem. Interview them. They'll tell you what tooling to buy. |
| Top AI Users (by distinct apps accessed) | Who's experimenting widest | High app diversity on one user is a training conversation, not an enforcement action |
| Applications-per-User breakdown | The shape of usage across the org | A long tail of one-app users plus a few ten-app users means two different policies |
| Personal versus enterprise account mix | Your actual data exposure | Personal-account usage on a top-3 app warrants Cloud Application Control this quarter |
| Dopamine DLP detections by type | Whether sensitive data is moving | Any PHI or PCI detection is a same-week action, regardless of volume |
Be honest with yourself about what these numbers are for. They're not a scoreboard for punishing employees. They're a procurement and risk input.
Trend versus spike: how to tell the difference
A spike is one user, one day, one tool. A trend is many users, several weeks, one direction.
The rolling 7-day window in AI Usage Analytics is deliberately short, which makes it good at surfacing change and bad at proving a trend on its own. So export the PDF weekly and keep the exports. Four consecutive weeks of the same report is a trend line you can put in front of a board.
Three patterns are worth naming:
- The onboarding spike. A new hire cohort starts, and AI requests jump. Predictable, benign, and a reason to include AI tooling in onboarding.
- The workaround migration. You block a tool, its transactions drop to zero, and a tool you've never heard of appears in Distinct AI Apps Detected the same week. That's not a win. That's displacement, and it's why we argue against blanket blocking in why blocking ChatGPT doesn't work and in how to detect shadow AI without blocking everything.
- The quiet climb. No single week looks alarming, but the 90-day slope is steep. This is the most common pattern and the one weekly-only reporting misses.
Building a reporting cadence that survives contact with reality
Pick three rhythms and stick to them.
Weekly, for the security team. Pull the AI Usage Analytics view. Look at Distinct AI Apps Detected for anything new. Look at Dopamine DLP detections for anything sensitive. Fifteen minutes.
Monthly, for the CISO. Export the branded PDF and write four paragraphs around it. More on what goes in that report below.
Quarterly, for the business. Take the top AI applications by users to the department heads who own those users. This is the conversation where you find out that marketing has been paying for a tool out of a credit card and that engineering wants Claude licensed properly.
What a monthly CISO report should contain
Six sections. No more.
- Headline numbers. Total AI Requests, Active AI Users, Distinct AI Apps Detected, with the prior month alongside each.
- Top 5 applications, by transactions and by users, flagged as licensed or unlicensed.
- Account exposure. How much of the top-app traffic runs on personal accounts rather than enterprise tenants.
- Data events. Dopamine DLP detections broken out by PII, PCI, PHI and IP, with the count blocked versus monitored.
- Changes since last month. New apps, retired apps, enforcement changes pushed.
- One recommendation. Not five. One thing you want a decision on.
The branded PDF export from AI Usage Analytics exists for exactly this audience, and handing a compliance lead a clean export beats handing them a CSV and an explanation.
Why the monitoring layer has to sit on the device
Shadow AI monitoring is only as good as what the sensor can see, and most sensors are placed somewhere that can't see enough.
DNS resolution tells you a device asked for a hostname. It cannot tell you the account, the prompt, or the file. Browser extensions cover the browser and stop at ChatGPT Desktop, Claude Desktop, Cursor and every CLI tool your engineers run. Cloud proxies add a detour of roughly 40 to 80 ms near a point of presence and 150 to 400 ms when users are far from one, and they can't inspect cert-pinned applications. CASB API connectors report on sanctioned SaaS after the fact. DSPM finds data sitting in storage, not data moving into a prompt.
dope.security runs the inspection on the endpoint. The dope.SWG agent decrypts and inspects locally in under 100 MB of RAM, identically on Mac and Windows, then traffic goes straight to its destination. That's the Fly Direct model, and it's why the telemetry behind AI Usage Analytics is complete rather than partial. We break the architectures down further in shadow AI tools compared.
One honest note. Sanctioned versus unsanctioned auto-classification and policy enforcement driven off these analytics are on the dope.security roadmap. They aren't shipped today. Today you get the visibility, the account-level control through Cloud Application Control, and the prompt-level control through Dopamine DLP, and you make the sanctioning call yourself.
Start monitoring this week
You don't need a committee to begin. Deploy the agent, let a week of telemetry accumulate, and read the first report. dope.security is $60 per device per year, listed publicly, with a free self-serve trial you start by signing in with Google or Microsoft. A Fortune 100 customer went from 900 devices to over 18,000 in weeks, deployed silently through Intune, so fleet size isn't the blocker you think it is.
Book a 20-minute demo and we'll show you the AI Usage Analytics dashboard against real traffic.
Frequently Asked Questions
What is shadow AI monitoring?
Shadow AI monitoring is the ongoing measurement of unapproved AI tool usage across your device fleet. It tracks which AI applications employees use, how many people use each one, whether they sign in with personal or corporate accounts, and what data goes into prompts and uploads. Unlike one-time discovery, monitoring runs continuously and produces a repeating report.
How do I monitor shadow AI usage without invading privacy?
Report on aggregates first and individuals only when a specific risk requires it. Track application counts, user counts and data classifications rather than reading prompt contents. Dopamine DLP classifies prompts using zero-retention APIs, so content isn't retained and no customer data trains a model. Publish what you monitor before you turn it on.
What metrics should I track for AI usage monitoring?
Track five core metrics: Total AI Requests, Active AI Users, Distinct AI Apps Detected, applications per user, and the split between personal and enterprise accounts. Add Dopamine DLP detections by data type (PII, PCI, PHI, IP). dope.security surfaces the first set in AI Usage Analytics over a rolling 7-day window.
How often should I report on shadow AI?
Weekly for the security team, monthly for the CISO, quarterly for department heads. Weekly catches new applications early. Monthly gives you enough data to separate a trend from a spike. Quarterly is where sanctioning decisions and budget conversations actually happen.
Can DNS filtering monitor shadow AI?
No, not adequately. DNS filtering resolves a hostname and stops there. It cannot read the path of an HTTPS request, cannot tell a personal ChatGPT login from a corporate one, and cannot see prompt contents. Roughly 95% of web traffic is encrypted, so a DNS-layer view of AI usage is a list of domains, not a picture of risk.
Does shadow AI monitoring cover desktop apps and CLI tools?
It does if the sensor sits on the device. ChatGPT Desktop, Claude Desktop, Cursor and CLI tools never touch a browser extension. dope.security inspects traffic at the OS level on the endpoint, so desktop and command-line AI usage appears in the same analytics as browser usage. See detecting shadow AI in desktop apps, IDEs and CLIs.
When should a shadow AI signal trigger action?
Act immediately on any PHI or PCI detection, on any new AI application that reaches your top five by users, and on personal-account usage of a top-three application. Everything else can wait for the monthly review. Acting on every signal trains your team to ignore the report.
What should I do once monitoring shows an unapproved tool is popular?
Sanction it or replace it, then control the account. Popularity is a signal that the tool does something your approved stack doesn't. Buy the enterprise tier, then use Cloud Application Control to allow the corporate tenant and block personal logins, which dope.security syncs across the fleet in under a minute.
.jpeg)
