How do you test an internal AI agent before you widen its access?
Short version. In a16z’s 28 October 2026 survey, 55% of CISOs said MCP servers or similar agent integrations were already in production, 46% named agents exceeding their permissions as a top worry, and 45% plan to buy internal AI agent security within twelve months. That budget line mixes three jobs: seeing agents on employee devices, governing what staff paste into outside tools, and testing an agent your company operates. Observ sells the third. One agent, tested through its normal interface as a user you provide, at $5,995, with a report, an attestation letter, a debrief, and a retest inside 90 days.
The same survey shows the tools are already in use. 92% report at least limited production use of employee generative AI. 83% report production use of coding assistants or agents. Security is already in the way of some of that rollout: 63% of responses describe reviews, approval requirements, or restrictions on particular uses. The worries they ranked are concrete. 47% include employees leaking sensitive data into AI tools. 46% cite agents exceeding their permissions or being compromised. 29% worry about prompt injection or data leakage through AI applications.
A security leader at a restaurant software company asked a16z the question underneath the category: “How do you really know what the agents are really doing?” An answer needs a boundary you wrote down, and a test of that boundary. A category name supplies neither.
Which problem is the budget for?
The budget is for one of three jobs. The survey’s single line, internal AI agent security, covers all three.
| Job | What good looks like | The product shape |
|---|---|---|
| Know which agents, extensions, and MCP servers run on employee devices, and whether a person or an agent took an action | An inventory, and a check at the moment of the action | An endpoint product built for agents. a16z’s point is that an agent using the employee’s own permissions looks like the employee to a last-decade endpoint tool |
| Stop staff pasting source, customer data, or secrets into a tool the company does not operate | A usage rule, a block, or a sanctioned tool | AI usage and governance. 38% plan a purchase or upgrade within twelve months. 47% already list data leaked into AI tools among their biggest worries |
| Learn whether an agent the company operates will expose data, call a tool it should not, or take an action outside the rules, when a test user drives the normal interface | A report with evidence, a fix, and a retest | An authorized test of that one agent. This is the Observ service, at $5,995 |
Buy the inventory when you cannot list the agents on laptops. Buy the governance control when the leak you fear is a person pasting into a product you do not run. Buy the test when you can point at one agent and you are about to give it another tool, a wider data set, or a larger set of users. 55% of these companies already have MCP servers or similar agent-to-application integrations in production, so a lot of teams are past the slide-deck stage on at least one of the three.
45% planning to “buy internal AI agent security” will be shown all three in one meeting. Ask which worry the product claims to close, and ask what evidence you will hold when the pilot ends. The endpoint problem is real. It is a different contract from a test of the support agent your team deployed.
What does an authorized agent test cover?
It covers one agent or LLM app you name, exercised as a test user you provide, through the interface a real user would use.
Observ prices that at $5,995, in US dollars. The work follows the OWASP Top 10 for LLM Applications. The classes are prompt injection, tool and permission misuse, data exposure, unsafe actions, output handling, and limits and cost. The report has an executive summary and technical findings. Each finding has a CVSS score where one fits, a location, the evidence, and the fix. A person on the team reviews every finding before it reaches you. You also get a signed attestation letter, a debrief with your engineers, and one retest of the findings you have fixed, if you request it inside 90 days. The full inclusion list is on the AI agent pentest page.
We turn down a third party, a model provider, a request for exploit code or a weaponized payload, and a production agent you will not schedule a window for. The form on the site asks us to contact you. The signed scope is what starts the test.
Put this in the scope, in plain language: the agent’s name, the interface, the test user, the tools it may call, the data it may see, and the actions that need a person to approve them. If those are unset, the window goes to discovering the boundary you meant to write down. “How do you know what the agent is doing?” starts as a sentence your own team can read. The test checks that sentence.
A host you also need evidence on is a separate engagement, at $3,995, described in the note on CVE volume. How a booked test sits next to a continuous platform is in the note on AI pentests and booked tests.
What should you have ready before you book?
Have a named agent, a test user, written authorization from someone allowed to give it, and a window.
Testing starts within 10 business days of the signed scope. The report follows within 5 business days of testing. If the agent is in production, schedule the window, or sign an acceptance of production risk. We decline the work when you will not.
An agent that can already call tools is ready for this test. The survey’s 55% figure is about integrations in production, which is that situation. A sketch with no user and no tools is early for a $5,995 engagement. The booking form is still the right first message. It does not reserve a window and it does not start a test. A deposit reserves the window. The scope starts the work. Those rules are also on the terms page.
Questions
We already have endpoint security. Do we still need an agent test?
An endpoint product answers which agents run on devices and whether a person or an agent took an action. An authorized test answers whether one agent you operate will expose data or take an action outside its rules. The a16z line “internal AI agent security” covers both purchases.
What does Observ test on an agent?
One agent or LLM app, through its normal interface, as a test user you provide. The classes are prompt injection, tool and permission misuse, data exposure, unsafe actions, output handling, and limits and cost, based on the OWASP Top 10 for LLM Applications. We do not test a model provider, and we do not test a third party.
How much does an AI agent test cost?
$5,995 for one agent, in US dollars. The fee includes the report, an attestation letter, a debrief, and one retest of the findings you have fixed, requested inside 90 days of the report.
Can you test every MCP server in the company?
The service is one agent you name. Listing every MCP server on employee devices is an inventory job. In the a16z survey, 55% already had MCP servers or similar integrations in production. Inventory and a test of the agent you operate are separate contracts.
What should we have ready before we book?
Written authorization, the name of the agent, a test user, and a window. Testing starts within 10 business days of the signed scope. We turn down a production target you will not schedule a window for.
Sources
- The New Security Agenda, Steph Zhang and Owen Colby, a16z, 28 October 2026. The adoption, worry, and purchase shares on this page are from that survey of 100 CISOs.
- OWASP Top 10 for Large Language Model Applications. Observ uses it as the basis for the agent test. The class list above is the one on our service page.
- Observ AI agent pentest. Price, classes, and what the fee includes.
Author
Keith Vaughan founded Cipher Projects in Canberra in 2023. Observ is its pentesting service. The survey figures are a16z’s. The $5,995 scope is ours. We sell a test of one agent you name. We do not sell endpoint inventory or a company-wide block on outside chatbots.