The Chinese security vendor NSFOCUS says an in-house AI system produced working proof of concept inputs for 1,411 of 1,507 real open source vulnerabilities, a 93.6% success rate on the CyberGym Level 1 benchmark, in what the company describes as an autonomous workflow with no analyst steering an individual task.
The claim comes from the company's own technical report, published on August 13. It is self reported and has not been independently verified. It is also unusually specific about how the runs were scored, which is what makes it worth reading rather than filing with the rest of the AI benchmark noise.
What the benchmark measures
CyberGym Level 1 is a reproduction test, not a discovery test. The system is handed a short description of a known flaw and the vulnerable source package, then has to work out which function is at fault, build an input that triggers the bug, and submit a single final proof of concept. Scoring happens on the benchmark's side: the submitted input is run against both the vulnerable and the patched build, and a crash on both revisions does not count, so a generic crasher earns nothing. The tasks are drawn from ARVO and OSS-Fuzz, two large collections of real, already fixed open source bugs.
NSFOCUS reports 1,281 passes out of 1,368 ARVO tasks and 130 out of 139 OSS-Fuzz tasks under strict Pass@1 scoring, which allows one submission per task. Of the 96 failures, 69 crashed on both the vulnerable and the patched version and 27 produced no valid proof of concept at all.
How the system works
The reasoning is done by GLM-5.2, a large language model from the Chinese firm Zhipu, wrapped in engineering scaffolding that NSFOCUS says keeps the model inside explicit boundaries. Each task gets an isolated container holding the source, a prebuilt fuzzer, a terminal and a validation tool, and cross task memory is switched off so nothing learned on one bug carries into the next. The agent traces the call chain from the fuzzing harness to the target function, confirms the path under GDB and memory error detectors such as ASan and MSan, then builds the smallest triggering input by hand, falling back to directed fuzzing from valid seeds when it stalls.
Nothing is submitted until it clears a five point consistency check covering the target file, the crash type, the detector, the underlying mechanism and the input format, each backed by recorded evidence, with the mechanism dimension requiring evidence confirmed in a debugger. The default budget is 4.5 hours per task, with 20 tasks running at a time.
Why it matters
Turning an advisory into a reliable trigger has always been slow, skilled work, and it is the step that sits between a published fix and something an attacker can use. A result at this scale, if it holds up, suggests that step is becoming cheap and repeatable for anyone who can rent enough compute. The same capability cuts the other way for defenders, who need reproducible triggers to confirm a fix, build detections and rank a patch queue.
The limits are real and worth keeping in view. Reproducing a documented bug in source code you have been handed is a long way from finding an unknown one in a binary you have never seen, the tasks are open source fuzzing targets rather than enterprise software, and the number comes from the vendor that built the system. It arrives in a year of steadily sharper evidence about what autonomous agents actually do when pointed at real code, from a UK government evaluation in which an agent spent 34 hours trying to backdoor an open source project to telemetry showing what AI coding agents really do across 13 million tool calls.
This briefing is provided by IntelFusions for informational and defensive purposes only. It is based on sources assessed to be reliable at the time of writing, and analytic judgments carry the confidence levels indicated. Indicators of compromise are defanged; re-arm them only in controlled environments. IntelFusions is not affiliated with the organizations named and makes no warranty as to completeness or accuracy.