Why test again when your systems haven’t changed?

A recruitment portal can work normally while an applicant account can read other people’s CVs. A photo store can complete checkout correctly while sending the original image to the browser before payment. These are failures in what the system allows, not necessarily in how its public pages look.

The useful security question is: what could someone do that they should not be allowed to do?

Two different reasons to test again

A software or infrastructure change can introduce a new path. A permission check may be lost when an API route is replaced, or a storage setting may make private files public. Comparing the previous and current versions with the same test accounts helps establish what changed.

A new AI model is a different trigger. The system may be unchanged, but a test may now connect clues that an earlier run missed. That does not make the vulnerability new. Nor does one successful run prove that the model update alone caused the discovery: access, tools, instructions, test budget and run-to-run variation all matter.

The UK AI Security Institute compared models released over eighteen months on a simulated corporate attack. Average progress rose from 1.7 to 9.8 steps with the same AI processing allowance. There were no active defenders. This shows progress in the controlled task, not a breach rate for real organisations. Read the evaluation.

Follow the path, then limit the claim

In an application test, a normal account is often a more useful starting point than a list of missing headers. Can it see another user’s record? Does that response reveal a document link? Does the download route check the same permissions?

A controlled test can demonstrate that chain using two seeded records. It should not silently turn “a second test record was accessible” into “the entire database was stolen.” The report needs to distinguish demonstrated access, possible wider exposure and actual misuse.

The same applies to payment workflows. Hiding a download button does not protect a file if the server has already returned a usable link. The fix must enforce the purchase requirement where access is granted, including storage and content delivery.

What the report should preserve

• The tested version, access level and relevant requests and responses.
• What an attacker could do, and where the proof stopped.
• The underlying permission or workflow failure.
• What prompted the test, and what supports any claim about why it succeeded.
• The fix and tests for both forbidden access and legitimate use.

These questions matter across applications, APIs and connected infrastructure, whether an organisation has its own security department or relies on a provider.

EverBreach is developing AI attacker simulations around this approach. The current external scanner has a narrower scope; the sample report uses clearly labelled fictional application scenarios to illustrate the intended depth. Discuss an assessment to agree what is available and appropriate for your systems.