Fewer than one in ten cybersecurity professionals trust AI testing tools to detect vulnerabilities, and more than three-quarters say their AI vulnerability scanning tools missed critical flaws.

Cobalt’s State of Pentesting 2026 report shows that confidence in fully automated AI testing has plummeted, from 29% in 2025 to 9% this year.
78% of respondents found that automated tools ignored critical vulnerabilities; The LLM flaws proved complex, with MTTR dropping from 19 to 36 days and most issues remaining unresolved.
Hybrid models have surged to 47% adoption, with experts emphasizing that automation should complement, not replace, elite human expertise in uncovering business logic risks.

As the world praises the Myth and the Chinese rush to create their own variant, a report painting a completely different picture comes from Cobalt.

The cybersecurity company just released the Cobalt State of Pentesting Report 2026, based on two comparative surveys, one in 2025 and one in 2026. By surveying around 450 cybersecurity professionals, Cobalt wanted to see how confident the cybersecurity community is in automated AI vulnerability testing and it turns out it’s not that much.

Last year, just under a third (29%) relied entirely on AI automation for testing. This year, that figure fell to 9%. Cobalt suggests that the main reason for such a drop in confidence is the fact that 78% of respondents found that fully automated scanning tools miss critical vulnerabilities. Another key reason is the complexity of the AI attack surface tested by the scanners.

Context-dependent vulnerabilities

Around one in three results from an AI test are deemed “high risk”, which is 2.7 times the average for conventional software, it was said. Additionally, at the time of analysis, less than two-fifths (38%) of LLM vulnerabilities were patched, while 62% remained open. The mean time to resolution (MTTR) for AI/LLM security issues increased from 19 days to 36 days.

“LLM vulnerabilities are deeply context-dependent and invisible to tools that lack an architectural understanding of the application,” said Andrew Obadiaru, CISO at Cobalt. “To close the validation gap, automation must be deployed exactly where it excels, but elite human expertise remains fundamental to discovering and remediating the most complex business logic risks. »

It took less than a year for the cybersecurity community to almost completely abandon fully automated AI testing and replace it with a hybrid model – about 47% said they now preferred it. This model grew 22% year-over-year, while the percentage of organizations using automation for low-risk environments also increased to 47%.

“While the industry is rightly excited about the potential of Mythos-class tools, unguided algorithms are inherently prone to returning even more false positives and costly false negatives than the automated scanners we have today,” Obadiaru continued.

Via Infosecurity Magazine