AI‑driven Security Testing: What South African Businesses Need to Know

South African companies are being offered continuous, AI‑driven security testing that promises lower costs and faster results. However, studies show that fully automated tools miss critical vulnerabilities, and only a hybrid approach—automation for breadth and human experts for depth—provides relia…

By Felo News Desk · Published Mon, 21 Sep 2026 21:54:23 GMT

Every South African business is about to receive the same pitch: continuous, AI‑driven security testing that is cheaper than the annual scans they are used to. Two of the three claims in the pitch hold up, but the third—whether the testing can truly protect the business—requires careful scrutiny.

What the Data Says About AI Testing

In June 2026, Cobalt released its AI and Pentesting Pulse Report, comparing two surveys of security professionals a year apart. In 2025, 29% of organisations relied entirely on AI automation for testing. By 2026 that figure had dropped to 9%. The research explains why: 78% of respondents reported that fully automated scanning tools missed critical vulnerabilities in their environments.

These findings are not about alert fatigue. The real problem is that a scanner can raise a false alarm and waste a week chasing it, or—more dangerously—remain silent about a real threat, giving the business a false sense of security.

Vulnerability vs. Attack Path

A vulnerability is a fact about a single system. A "way in" is a sequence of steps an attacker could take. Automation excels at identifying individual vulnerabilities, but it struggles to chain them into a realistic attack path—an exercise known as Adversary Path Engineering. The difference matters because attackers do not exploit a single severity rating; they follow a route that connects multiple weak points.

Human Validation Still Matters

In a hacking contest run by Hack The Box in November 2025, AI‑assisted teams solved 36 problems at a rate 3.2 times faster than non‑AI teams. However, among the top 5% of teams, the gap narrowed to 1.69 times, and the best AI‑assisted team stopped at 32 problems while the top human team solved all 36. The less skilled the tester, the more automation helps; the more skilled, the less difference it makes.

Today, 47% of providers use a hybrid model: automation for breadth and qualified operators for judgment. According to Marthinus Engelbrecht, group chief executive of NEWORDER, “Automation is used where it is genuinely better—breadth and repetition. Every finding is then confirmed by a qualified operator before it reaches a client.” This process, known as Human Validation at Scale, transforms a simple report into a true assessment.

Why Continuous Testing Is Attractive—and Risky

South Africa faces a talent gap: the CSIR found that 63% of cyber security roles are empty or only partially filled. Continuous testing without a scarce specialist appears to be a solution. But if a test misses the route an attacker would use, the report will look clean, giving a false sense of safety. An empty report is easier to accept because it costs less, arrives faster, and says nothing is wrong. Yet a test that finds nothing and a test that misses everything produce identical documents.

Regulatory Pressure and What Buyers Should Demand

King V, effective from 1 January 2026, places governance of data, information, and technology on the board. Companies must be able to explain what was tested, by whom, and what was left out. A certificate or accreditation is useful but does not describe the work performed. Five key pieces of evidence can help buyers assess a provider:

Documented methodology of the test
Identity of the person who ran it
Attack path discovered and its endpoint
Proof that a human confirmed each finding
Retest showing the path is closed

NEWORDER, founded in 2010 and ISO/IEC 27001 certified, claims to provide all five. Engelbrecht advises buyers to ask for these details before signing. Providers who cannot supply them likely rely solely on automated systems.

Three Questions Before You Sign

What did the AI tool do, and what did a human do?
Did the provider chain findings into an attack path or merely list them?
Who bears the cost if the test misses a real risk?

Answering these questions reveals whether a provider truly believes in the quality of its product. If a provider cannot draw the line between automation and human work, it may be selling a document rather than a test.

In short, while AI can accelerate vulnerability discovery, it cannot replace the strategic insight that comes from human experts mapping attack paths and validating findings. South African businesses should look for a hybrid approach that balances speed, breadth, and depth to ensure real protection.

Key facts

AI testing can miss critical vulnerabilities, especially attack paths
Hybrid models—automation plus human validation—provide the best coverage
Continuous testing is appealing but can give a false sense of security
Buyers should demand evidence of methodology, human confirmation, and retesting
Regulatory changes require clear documentation of what was tested

Why it matters

AI‑driven security testing offers speed and cost savings, but without human oversight it can miss the very routes attackers use, leaving businesses exposed. Understanding the limits of automation helps companies choose a truly effective security strategy.