Skip to content

Response, New York. Noah Park: Check Point Patches Actively Exploited Management Server Path Traversal. Detection, Dallas. Ava Okello: Report Finds Nearly Half of Deployed Detections Need Attention Before SOCs Can Trust Them.

SOCtember

Always First. Fast SOC News.

Architecture diagram from arXiv paper 2611.19295. Inside a dashed box labelled internal Docker network, no published ports, an attacker container sends app HTTP to a V2 gateway and a synthetic login to an OpenCanary sensor. A legitimate client sends a valid key to the gateway, which fronts a mock Records API. The sensor sends a webhook from a verified source to an adapter that verifies, redacts and signs, then posts with HMAC to a control endpoint that tells the gateway to revoke the peer for 15 minutes. The adapter also feeds a collector of redacted evidence.
Figure: Raditio Ghifiardi, arXiv:2611.19295 (CC BY 4.0), Fig. 1, the lab's V2 architecture, rendered from the PDF and placed on a white background.

Opinion

A Lab Study Found Canary Credentials Caught LLM Attack Agents in 19 of 20 Sessions but Prevented Nothing That Hardening Had Not Already Stopped, and SOC Teams Should Budget Deception as Detection

An independent researcher pitted small LLM agents against a mock API with and without decoys, and found that agents skipped traps placed beside a real weakness, that fixing the weakness did all the preventing, and that a planted canary key turned into fast, client-scoped revocation. Elena Vos argues that SOC teams should put bait on the attacker's path, wire it to response and stop counting it as a control.

Elena Vos, OpinionLondon6 min read

LONDON - Deception has always sold itself on one clean promise: nobody legitimate ever touches a fake asset, so any touch is an alert worth acting on. Security teams are now being told that the promise gets even better against AI attackers, because a language model agent reads and acts on whatever text a system hands back. A small, careful study posted to arXiv on Nov. 14 tests that idea against a control group, and its answer should change how SOC teams justify and wire their decoys.

Raditio Ghifiardi, an independent researcher in Jakarta, built an isolated lab around a mock Records API with one deliberate weakness: a debug endpoint, /api/v1/debug/env, that leaks the real service key. The attacker was an LLM agent that could read any response and use any credential it found. Every agent and defense event went into a SHA-256 hash chain that a separate verifier recomputed. All targets and credentials were synthetic, and the models were small local ones of 4B and 9.7B parameters. The paper says plainly that "all results are lab-scale and are not a production claim." Read with that limit in mind, the four studies still make a sharp point.

Screenshot of the arXiv abstract page for paper 2611.19295, Optional Traps: An Empirical Study of LLM Agents Against Credential Deception and Alert-Driven Revocation, by Raditio Ghifiardi, submitted on 14 Nov 2026 under Cryptography and Security, with the abstract and links to the PDF and the code and evidence archive.
Image: arXiv (fair use).The abstract page for arXiv:2611.19295, captured Nov. 19, 2026.

Traps beside the path get skipped

In the first study, a 9.7B-parameter model faced two deception setups. One added a backup file holding a canary key and a deprecated internal API advertised in robots.txt and the OpenAPI document. The other made that internal API return endless pagination leads. In both, the agent ignored every decoy, read the genuine key from the debug endpoint and reached the export in 6 HTTP calls. No containment fired. The paper treats this as one session per mode, not a comparison, but calls it "a concrete counter-example" to the optimistic view, adding that "when a genuine weakness is reachable, decoys placed beside it can simply be skipped."

That matches a pattern defenders should recognise. Deception thinking often assumes the intruder explores. As the paper puts it, "a goal-directed agent with a working path does not need to."

Hardening did the preventing

The redesign moved the bait onto the path. The second study ran 60 sessions with a 4B-parameter model under three conditions: a baseline where the debug route leaks the real key, a hardened setup where it returns 404, and a defended setup where it serves a planted canary key and any use of that key revokes the client for 15 minutes. Genuine exports went from 20 of 20 at baseline to 0 of 20 with hardening alone and 0 of 20 with hardening plus deception. Baseline against defended was significant, with an exact McNemar p of 1.9 x 10^-6. Hardened against defended showed no difference at all, p = 1.

Table II from arXiv paper 2611.19295, Study 2 results, all 60 sessions valid. Baseline, debug route leaks the real lab key: genuine export success 20 of 20, 100%, Wilson 95% interval 83.9% to 100%, peer revoked 0 of 20. Hardened, returns 404: 0 of 20, 0% to 16.1%, peer revoked 0 of 20. Defended, serves a planted canary key with enforcement on: 0 of 20, 0% to 16.1%, peer revoked 19 of 20.
Figure: Ghifiardi, arXiv:2611.19295 (CC BY 4.0), Table II, rendered from the PDF and resized on a white background.

What deception added was the last column. The defended setup identified and revoked the attacking client in 19 of 20 sessions, something the hardened setup by design could not do. In the one exception, the model read the bait route, presented a credential the export did not recognise, got an HTTP 401 and then ran out of turns. "A defender who removes the leak gets the prevention; adding deception adds a signal," the paper concludes.

The alert can drive a narrow response

The third study is the one a SOC engineer will want to copy. On a Google Cloud VM, a real OpenCanary login alert was accepted only from the verified sensor address, redacted, signed with a per-run HMAC secret and posted to a control endpoint that rejects stale, forged and replayed events. The gateway then revoked that one client even though it held a valid token. The paper's event chain shows OpenCanary logging the login at 08:14:17.957 UTC, the gateway logging the revocation at 08:14:17.970 and denying the export at 08:14:18.046. Measured from the start of the login, the first denied export came within an observed upper bound of 303 ms, while a separate legitimate client completed 100 of 100 requests. All 14 control checks passed, including a forged sensor webhook (403), a spoofed forwarded identity after revocation (403) and a replayed signed event (409).

Table III from arXiv paper 2611.19295, control checks in the final cloud run, 14 of 14 passed. Export without a credential 401; legitimate client before detection 200; attacker with supplied valid token before detection 200; forged sensor webhook from the attacker 403; replay of the V1 debug-key bypass 403, revoked; real OpenCanary login from the attacker 200, alert; same attacker with same valid token after the alert 403; spoofed forwarded identity after revocation 403; legitimate client after revocation 100 of 100 OK; invalid HMAC signature 401; signed control event 202; replayed signed event 409.
Figure: Ghifiardi, arXiv:2611.19295 (CC BY 4.0), Table III, rendered from the PDF and resized on a white background.

The author is careful that 303 ms and an earlier 266 ms are "two single observations, not a latency distribution." The design lesson does not depend on the number, though. The response was safe because of a verified sensor source, a short validity window, replay protection and a client identity the attacker could not spoof.

LLM clients break the false-positive rule

The fourth study, run as single swarm engagements, delivered the most uncomfortable finding. Agents that used the canary were revoked 39, 54 and 59 seconds after the bait was served, without affecting the other agents. But a legitimate LLM agent that held the correct key in its prompt still fetched the debug route, preferred the "leaked" credential, used it and was revoked. "For deception aimed at LLM clients, bait that looks useful can also be taken by benign automation," the paper warns.

A separate paper posted on Nov. 17 by researchers at the University of Murcia, SCORPION Cybertech and the Technical University of Denmark looks at the other side of deception against AI attackers: keeping them engaged. Across 1,150 sessions of LLM attackers against an LLM-driven SSH honeypot, coherent scenarios of linked files, processes, services and logs raised the median session length of advanced attackers from 40 to 165 commands and cut their abandonment from 86% to 52%. Nearly all sessions ended through loss of interest rather than the attacker spotting the trap. Both papers point the same way: against agents, deception earns its keep as intelligence and detection, not as a wall.

What SOC teams should change

  • Put bait where the attacker has to look. A canary key next to a real leak is optional; a canary key in place of the leak was used in 19 of 20 sessions.
  • Fix the weakness first and count the fix as the control. In this study, hardening alone matched hardening plus deception on prevention, 0 of 20 exports each.
  • Budget and report deception as detection. Its measured value here was fast, attributable alerts and client-scoped revocation, which a hardened system cannot give you.
  • Wire canary alerts to response with guardrails: accept events only from a verified sensor, sign them, reject stale and replayed ones, and revoke by an identity the client cannot set, not X-Forwarded-For.
  • Treat a canary hit from an AI-driven integration as high-signal but not conclusive. The paper recommends reversible, scoped responses, such as its 15-minute per-client revocation, over blanket blocks.
  • Read every "zero exports" result with its reason. In several runs, small models failed on wrong API paths or exhausted budgets, which the author says reflects "a capability miss as much as the defense."

None of this makes deception less useful. It makes it honest. The study's own summary is the right one to put in front of a budget committee: "closing real weaknesses does the preventive work, and deception's measurable value is fast, attributable detection."

About the Author

Elena Vos

Elena Vos covers SOC strategy and detection operations for SOCtember from London.

Related stories

Opinion desk