Skip to content

Response, New York. Noah Park: Check Point Patches Actively Exploited Management Server Path Traversal. Detection, Dallas. Ava Okello: Report Finds Nearly Half of Deployed Detections Need Attention Before SOCs Can Trust Them.

SOCtember

Always First. Fast SOC News.

SecOps Unpacked hand-drawn ASEF AI SOC Evaluation Framework diagram showing the Screen, Score, Platform and Trust, ROI and Decide stages.
Diagram: SecOps Unpacked / Filip Stojkovski, ASEF framework from the Introducing ASEF post, July 8, 2026 (fair use).

Opinion

AI SOC Buyers Need a Lifecycle Score, Not Another Middle-Heavy Demo Checklist

SecOps Unpacked's ASEF framework (July 8, 2026) argues the AI SOC market optimized for triage and investigation demos while detection engineering and response stay thin, and buyers need a vendor-neutral lifecycle profile that refuses blended scores, treats unknown as unknown, and scores trust and feedback loops.

Elena Vos, OpinionLondon4 min read

LONDON - The AI SOC market did not fail because demos are weak. It failed buyers because the demos cluster in the middle. Filip Stojkovski's SecOps Unpacked post introducing ASEF, the AI SOC Evaluation Framework, published July 8, 2026, puts the clustering in plain language: triage and investigation were the easier win, products piled up there, and both ends of the lifecycle stayed thin. Very few products improve detections. Very few execute remediation without heavy human oversight. The label "AI SOC" still covers everything from a password-reset chatbot to a full investigation pipeline, with no shared way to measure the distance.

SecOps Unpacked banner, Independent SecOps Research, on a dark desk canvas.
Photo: SecOps Unpacked, official banner on a SOCtember desk canvas; the ASEF guide is at secops-unpacked.ai/asef/guide (fair use).

ASEF is Stojkovski's vendor-neutral answer. It scores platforms across data ingestion, detection engineering, investigation and triage, and response with a feedback loop, plus a cross-cutting Platform and Trust layer that never blends into a headline feature score. The interactive guide at secops-unpacked.ai/asef/guide, now at ASEF v3.2.0, frames the work as a Shift Map that runs from a data foundation through detection and readiness and investigation and understanding to remediation and improvements, with Platform and Trust spanning every zone. Buyers can score the whole map for fullness, or scope to the zones they are actually buying and score those in depth with no penalty for the rest. Out-of-scope zones read as not evaluated, never as silent gaps.

SOCtember illustration of a Middle-heavy market: thin data, detection and response ends around a demo-heavy investigation and triage middle.
Illustration: SOCtember, conceptual Middle-heavy market: thin ends, demo-heavy middle.

As Stojkovski laid it out in July, the framework refuses the evaluation habits that let Middle-heavy products look mature. Requirements stay binary filters; they are not scores. Platform and Trust sits on its own card; a platform score below 50 percent raises a risk flag that feature strength cannot clear. The headline is a profile across chosen zones, not one blended number. A product that is expert at investigation and empty on both ends is labeled Middle-heavy on purpose. Autonomy is scored on a scale carried over from ARMM, running from 0 through three assisted levels (1C, 1G and 1A) to 2, end to end with no human in the loop, so coverage without depth and real autonomy where it counts read as different products for different buyers.

SOCtember illustration contrasting a single blended score with a gated zone profile labeled Middle-heavy.
Illustration: SOCtember, conceptual contrast between a blended score that hides weakness and a gated zone profile.

Screening integrity is the rule Stojkovski says he cares about most: a vendor fact is present or it is unknown. Unknown never quietly becomes a pass or a fail. Vendors land as passes on known facts, excluded when a known fact fails a hard requirement, or unknown when a required fact is missing and must be verified in a proof of concept. The same honesty applies to the model. Inconclusive verdict support is a scored Investigation capability, because forcing a binary call on thin evidence is reckless, not confident. Override telemetry and published failure modes live in Platform and Trust. ROI is a PICERL panel of deltas against a buyer baseline across SANS incident-response phases, not a single speed number. Closing alerts faster is sweeping the floor faster. The question ASEF keeps asking is whether the SOC is getting smarter. The current v3.2.0 guide has since reorganized each capability check around three separate answers, what ships, who does the work and what evidence backs the claim, and it still keeps missing information as unknown rather than a zero score.

SOCtember illustration of screening states: Passes, Excluded and Unknown.
Illustration: SOCtember, conceptual screening integrity: Passes, Excluded and Unknown; unknown is not a pass.

That framing lines up with practitioner skepticism already on the record. In March 2026, Help Net Security covered Anton Chuvakin and Oliver Rochford's "When Marketing Fails" paper, drawn from more than 30 vendor briefings plus practitioner commentary. Their finding: most claims describe a future state sold in the present tense, and when reality disagrees, product immaturity is reframed as a buyer readiness problem. Their fix is to separate assistive from autonomous capabilities, measure adoption depth instead of feature enablement, and treat trust as something a system earns. ASEF builds mechanisms for each of those points into the scorecard rather than leaving them as slide language.

This thesis is distinct from the other SOCtember Opinion lines it sits beside. The alert fatigue piece treats noise as a detection-pipeline failure, not an analyst character flaw. The Splunk ES 8.7 piece treats trusted autonomy as governance, not a model upgrade. The Anvilogic piece asks why teams measure false positives without owning the cuts. The Ponemon and Crogl piece asks why most alerts never become investigations even when AI promises speed. The SANS piece treats retention and AI adoption as operations-design problems. The Binalyze piece sharpens the 37-versus-28-minute race and the investigation-ready evidence gap. This ASEF piece is a seventh claim: buyers need a vendor-neutral lifecycle score that refuses Middle-heavy demo checklists, blended numbers, and unknown-as-pass scoring, and that puts trust, inconclusive handling, and feedback loops on the scorecard.

Sources:


Elena Vos covers SOC strategy and detection operations for SOCtember from London.

Related stories

Opinion desk