An AI Analyst That Names the Right Attack Stage for the Wrong Reasons Is Not Ready to Drive Incident Response

LONDON - The pitch for AI in incident response has moved past triage. Agents are now offered that read an incident's logs and tell responders how far the attacker got: still probing, running code, or already taking data. That judgment decides whether a team patches, isolates a host or starts a breach notification. A new benchmark suggests SOCs should check not only whether the agent's answer is right, but whether the logs it cites actually support it.

The preprint, SecLog-Bench, was posted to arXiv on Oct. 22 by Yifan Wang, Dongliang Xu, Xiaoyong Li and Yue Yao of Shandong University, Jing Li of the University of Technology Sydney and independent researcher Peter S. Pan, and has been submitted to IEEE Transactions on Dependable and Secure Computing. The team built 29 reproducible attack scenarios, 25 of them based on CVEs, including Sandworm-style simulations and a multi-hop attack in a 26-node service cluster with ten inter-host hops. From each run they collected host audit logs, application logs and network traffic, then cut the results into 216 human-reviewed evidence windows, 54 for each of four labels: PREPARATION, EXECUTION, RESULT and STAGE UNCLOSED, the label for windows where the visible evidence is not enough to name a stage. Each window carries a four-step reference reasoning chain, 864 annotated steps in all.

Eight models were tested between May 21 and June 27: Qwen3-8B, Qwen3-32B, Llama-3.1-8B-Instruct, DeepSeek-R1-0528, Qwen3.6-Plus, Gemini 3.1 Pro Preview, Claude Sonnet 4.6 and GPT-5.4. Every window was already attack-related, so no model had to find the attack first. Each only had to say how far it had gone and cite the evidence.

A right label is not a right answer

Claude Sonnet 4.6 scored best, with 44.44 percent accuracy and 45.07 percent Macro-F1. That beats the 25 percent a random guess would reach across four labels, but as the authors put it, "fewer than half of the evidence windows receive the correct stage label."

The more important result sits underneath. The benchmark also checked whether each answer's cited reasoning held up against the logs. Only 41.67 percent of Claude Sonnet 4.6's correct answers came with reasoning that met the paper's reliability criteria, and its reasoning score, which the authors call Stage Reasoning F1, was 38.67 percent. On EXECUTION windows it reached 48.15 percent recall, yet none of those correct EXECUTION calls was backed by reliable reasoning. In one example it correctly labeled a template-injection request followed by process forks as EXECUTION, but "incorrectly treats the forks as direct evidence that injected commands were executed, although no such commands are visible."

GPT-5.4 showed the opposite profile. Its accuracy was 33.33 percent, mid-table, but it posted the highest reasoning score, 54.87 percent, and 81.94 percent of its correct answers were backed by reliable reasoning. Across all eight models the Pearson correlation between accuracy and reasoning quality was weak, at 0.15. The authors' own conclusion is that "final-label accuracy alone can overstate model capability."

The failures are the ones responders fear

The errors clustered in the two judgments that matter most after an alert is confirmed. Models routinely missed visible consequences such as responses, returned data or state changes. The share of RESULT windows given some other label ranged from 53.7 percent to 92.6 percent across the models, and RESULT had the lowest mean recall of any label, 25.46 percent. The models also over-committed. On windows where the evidence was insufficient, every model assigned a concrete stage at least half the time, rising to 92.6 percent for Llama-3.1-8B-Instruct.

In a SOC those are opposite errors with the same root. Missing a RESULT means telling the business nothing left the building when the logs show that something did. Forcing a stage onto thin evidence produces a confident scoping statement that nobody can defend in the post-incident review. Both come from failing to judge whether the visible evidence is sufficient, which is the core skill of an incident analyst.

More logs made it worse

The obvious fix is to hand the model everything. The researchers tested that with GPT-5.4 on a subset of 96 windows. Fed raw, uncompressed logs from all three sources, its reasoning score was 2.77 percent. Given the same sources compressed and organized into a scaffolded evidence view, its accuracy rose from 27.08 percent to 50.00 percent and its reasoning score reached 44.09 percent. The authors argue that input design should "prioritize evidence traceability and clear cross-source relations, rather than simply expanding context length."

That matters for any team wiring an agent straight into a SIEM and letting it pull whatever it likes. In this study the quality of the evidence package, not the size of the context window, decided whether the reasoning held.

What SOC leaders should require

• Every stage or scope claim cites specific log events, and an analyst can open each one.
• Insufficient evidence is an expected answer. An agent that never says so is a warning sign, not a strength.
• Evaluations score the reasoning chain against the evidence, not only whether the final label matched.
• A human checks consequence evidence, such as returned data, file writes and outbound transfers, before anyone tells the business an attack failed.
• Evidence is correlated across host, application and network sources and organized before the model sees it, rather than dumped in raw.

The study has limits the authors state plainly. Its 216 windows were selected as attack-related, so it does not measure attack discovery, benign-versus-malicious discrimination or operational false-positive rates, and its balanced labels do not reflect the class mix of a real queue. Hosted models may also change with updates. The authors see a role for the models in organizing facts across log sources and exposing evidence gaps, but add: "Their predictions and reasoning outputs still require human review."

The paper describes inferring attack progress from logs as "not merely a problem of classifying attack stages, but a security reasoning task." That is the standard SOCs should hold AI to. A right answer reached through the wrong evidence is a wrong answer waiting for the next incident, delivered with the same confidence.