Should you buy an AI pentest or book another test?
Short version. a16z asked 100 CISOs at companies with more than 1,000 employees about 2027. 42% plan to buy or upgrade offensive security within twelve months. 13% report production use of either AI offensive security or AI exposure and vulnerability remediation. 18% say most offensive testing already runs continuously. 52% expect that within a year. Those figures describe a market of pilots. Book an authorized test when you can name one host or one agent and you need a person to confirm the findings and retest the fixes. Buy a continuous platform when you can run it on releases and staff the queue.
Steph Zhang and Owen Colby published The New Security Agenda on 28 October 2026. The piece reports the survey, and it is also an investor’s map of categories, including companies the firm has backed. Read the percentages as what those CISOs said. Read the product mentions as the firm’s view of the market. This note is the buying rule we would use, with our prices stated where the rule points at us.
What did CISOs say they will buy?
They said they will spend more, keep or grow the team, and put AI into the security stack, while most have not put AI offensive security into production.
87% expect cybersecurity spending to increase, driven by AI-enabled threats and by the attack surface of their own AI deployments. Nearly half expect security headcount to rise in 2027, about half expect it to stay flat, and 4% expect cuts. The extra budget is aimed at tools, and the headcount answers say the team stays.
Asked to recall 2026, exposure and vulnerability management and SIEM sat at the top. For 2027, the initiatives they selected involve AI. SOC automation drew the most first-place votes. Internal AI agent security, SOC automation, and offensive security gained the most ground. Code security, remediation, and AI governance followed. Where a team starts depends on where the pressure is.
| What they reported | Share |
|---|---|
| Plan a new purchase or upgrade of offensive security within 12 months | 42% |
| Production use of either AI offensive security or AI exposure and vulnerability remediation | 13% |
| More frequent security posture testing is important (54% said very important) | 83% |
| Most offensive testing already runs continuously against production assets | 18% |
| Expect most offensive testing to run continuously within a year | 52% |
| Expect testing embedded in CI/CD within five years | 44% |
| Want a retest that verifies a fix | 70% |
| Want remediation ordered by validated exploitability | 67% |
The 13% line is one bucket in the write-up: “production use of either AI offensive security or AI exposure and vulnerability remediation.” We are not splitting it into two separate 13% figures. The purchase intention for offensive security is stated on its own, at 42%.
What does continuous mean in this survey?
Continuous, in the answers, means most offensive testing runs against production assets on an ongoing basis. 18% said that is already true. 52% expect it within a year.
That is a different question from whether a single annual test is enough. A team can test several times a year and still answer that most of its testing is not continuous. A CISO at a global manufacturer told a16z: “With all the new AI threats, we need to find things before others do.” The wish is the 83%. The installed base is the 18%.
Buyers will also hear a louder line. Intruder’s 2026 report, The state of AI pentesting in 2026, says 90% of its respondents agree the traditional annual pentest is dead, and that 6% rely on a single annual test. The two surveys do not ask the same people the same question. One asks large-company CISOs whether most testing is already continuous. The other asks whether they agree a claim about the annual test. Use 18% and 52% when you are deciding what is installed. Use 90% when you are deciding what a room is ready to say yes to.
When is a booked test the purchase?
A booked test is the purchase when you can name one host or one agent, a customer or an auditor wants a report, and you want a person to confirm each finding and retest the fixes.
GMO Internet showed what a slow fix costs when the company already has testers. It runs Onamae.com Desktop Cloud, hosted Windows desktops used for foreign-exchange trading, and has said as many as 26,739 of those desktops may have been affected in an October 2026 breach. Masatoshi Kumagai, the CEO, wrote that the company’s own testers had already pointed out improvements, and that the response “lacked sufficient depth and speed.” The Register quoted the blog on 27 October 2026. He went on: “The ease with which open weight AI models can be combined and spread into countless attack tools has transformed cyberattacks, which once required advanced skills, into an era where anyone can easily carry them out.” He said GMO’s security team had tied unauthorized access against about 20 Japanese companies to misuse of ARTEX, an autonomous testing tool built for defense, and that the group had asked Japan’s government to apply proactive cyber defense. He also wrote that the company was not asking the government to carry the blame for its own delay.
The government request is a national project. The purchase in front of a team that can name one host is the finding the testers already wrote down, closed, and retested. The open-weight line is why that finding cannot sit through another release cycle.
The survey also records why pilots stall. A CISO at a global apparel company had narrowed seven or eight vendors to three and said it was not humanly possible to proof-of-concept all of them. Another security leader had signed a short contract and was still evaluating, because he did not want a new point product for every new problem. a16z points to a Black Hat 2026 vendor landscape in which some categories list as many as 43 companies. Sorting that field is a project. Naming one host is a scope.
Observ is built for the named scope. Prices are public, in US dollars. A host is $3,995. An agent or LLM app is $5,995. Each fee includes the report, a signed attestation letter, a debrief, and one retest of the findings you have fixed, requested inside 90 days. A person reviews every finding. A deposit reserves a window. Written authorization starts the test. The letter states what was tested, when, and the result, for an auditor or a customer’s review. Observ is not a SOC 2 or ISO 27001 auditor, and the letter does not certify a framework.
A separate study is why the review step is a person. Synack’s State of Continuous Security Validation (June 2026, 97 enterprise security leaders, research Synack commissioned) found that 79% would not act on an AI finding until a person confirmed it. That is their measurement. It matches the way we ship a report. Our team uses AI tools for the recon and checking a day-rate firm bills as time, and a person still confirms the finding.
| The question you can already write down | What to buy | Where |
|---|---|---|
| What is reachable on this one internet-facing host, with evidence a customer can read? | Authorized host test, $3,995, retest included | Book a host test |
| What will this one agent do when a test user drives it? | Authorized agent test, $5,995, retest included | Book an agent test |
| What changed on every release, across the estate, and who triages it each week? | A continuous testing product, after a pilot the team can finish | Your incumbent platform, or a vendor you can actually evaluate |
| Which alerts should an analyst see, and which low-risk actions can an agent take? | SOC automation, the category with the most first-place votes | A SOC product. Observ does not sell one |
When is a platform the purchase?
A platform is the purchase when the asset list is the environment, a release has to trigger the test, and someone on your team owns the queue every week.
44% of these CISOs expect that shape inside CI/CD within five years. 52% expect most offensive testing to be continuous within a year. If a contract you are signing this quarter requires a test on every release, a single booked window is the wrong shape, and we would say so before a scope. Observ does not run inside a pipeline on every commit.
44% said they would look first to an existing security platform for AI security features. 21% would start with an AI-native vendor. 47% said they would choose a startup when it covers a gap the incumbent does not. A booked test fits that gap for a team whose platform does not produce a scoped, human-reviewed report on one host. It is a weak substitute for the platform they already trust with endpoints and telemetry. One leader in the piece put the incumbent advantage plainly: brand, and a procurement path that has already been walked.
What else moved for 2027?
Spend is up, teams stay, and the other first fires are SOC automation, the AI employees already use, and the volume of code.
- SOC automation is in production for 18%. Today, 36% are comfortable letting agents act on their own for low-risk actions. 60% expect to be comfortable with that within twelve months. 7% expect to allow autonomous action in most cases. Gathering evidence and taking an action that can stop the business are different permissions. We do not sell a SOC. On a test, the same split shows up as a written scope and a person signing the finding.
- Internal AI agent security is the other fast climb: 45% plan a purchase or upgrade within twelve months. That phrase mixes three products. The split is in the note on testing an agent you operate.
- Code security moved into the top five priorities for nearly half of respondents, from about a quarter in the 2026 priorities they recalled. A financial-services CISO told a16z: “We have way more engineers developing than we have security people testing.” A one-host test is how a small security team spends a window on the application those engineers shipped. It does not review every dependency in the company. The patch side of that pressure is in the note on CVE volume.
Questions
Is the annual penetration test over?
In the a16z survey of 28 October 2026, 18% of CISOs said most of their offensive testing already runs continuously, and 52% expect that within a year. Many customers still ask for a test on a calendar. What has changed is that one test is weak evidence for a year of releases.
Should we buy an AI offensive security platform in 2027?
Buy one when you can name the assets, finish a pilot, and staff the findings. In that survey, 42% plan to purchase or upgrade offensive security within twelve months, and 13% report production use of either AI offensive security or AI exposure and vulnerability remediation.
What does a booked Observ test include?
A host test is one host you name, at $3,995. An AI agent test is one agent or LLM app, at $5,995. Each includes a report, an attestation letter, a debrief, and one retest inside 90 days. A person reviews every finding. Written authorization starts the test.
Will Observ run inside our CI pipeline on every commit?
No. That is a continuous product. a16z found 44% of CISOs expect testing embedded in CI/CD within five years. The Observ service is a booked window on one host or one agent.
Does a person check findings that came from AI tools?
Yes. A person on the Observ team reviews every finding before the report ships. A June 2026 Synack study of 97 enterprise security leaders found that 79% would not act on an AI finding until a person confirmed it. That figure is Synack’s, from research they commissioned.
Sources
- The New Security Agenda, Steph Zhang and Owen Colby, a16z, 28 October 2026.
- State of Continuous Security Validation, Synack, June 2026. The 79% confirmation figure is theirs.
- The state of AI pentesting in 2026, Intruder. The 90% and 6% figures are theirs.
- Japanese virtual desktop cloud hacked so hard it asks to be banned, Simon Sharwood, The Register, 27 October 2026. The English wording of Kumagai’s blog is taken from this report.
Author
Keith Vaughan founded Cipher Projects in Canberra in 2023. Observ is its pentesting service. The survey figures on this page are a16z’s, Synack’s, or Intruder’s, as cited. The Kumagai lines are his, via The Register. The prices and the scope rules are ours.