ATM Architecture Demo Test

01Introduction: From Paper to Code

On April 29, 2026, CVE-2026-31431 (“Copy Fail”) was publicly disclosed. This is a local privilege escalation vulnerability in the Linux kernel’s authencesn cryptographic template that achieves a controlled 4-byte write to the page cache of any readable file through the interaction of the splice() zero-copy mechanism with AF_ALG sockets. The vulnerability had lain dormant since 2017, was discovered by Theori researcher Taeyang Lee, and was extended into a complete exploitation chain by the Xint Code team through AI-assisted analysis.

What drew our attention to Copy Fail’s discovery was not the vulnerability itself, but the fact that its discovery method was structurally isomorphic to the ATM methodology paper previously published by the LEECHO Research Lab (“Abductive Tracing Analysis of the 0-Day Bug Discovered by Mythos”): a human researcher provides directional insight (“AF_ALG + splice() exposes page cache references”), AI conducts directed scanning within the narrowed search space, and the vulnerability is hit within approximately one hour. This prompted our decision to codify the ATM methodology, build a runnable prototype tool, and validate its effectiveness on real targets.

02ATM Scanner Architecture

ATM Scanner is a React-based single-page application that automates ATM’s five-step process via the Claude API. Each step’s output is chained as context input to the next, forming a causal reasoning chain.

2.1 Five-Stage Pipeline

2.2 Technical Implementation Notes

The Scanner calls the model via the Claude Messages API in streaming mode (SSE). Two streaming parsing bugs were encountered and fixed during development: (1) TextDecoder was not initialized with stream: true mode, causing UTF-8 multibyte Chinese characters to be truncated at chunk boundaries; (2) SSE data lines could be split across chunks, causing JSON parsing to silently fail and lose content. A line-buffering mechanism was added post-fix to ensure only complete SSE data lines are parsed.

For model selection, the Scanner offers three tiers: Opus 4.6, Sonnet 4, and Haiku 4.5, with an adjustable max_tokens slider (2K–16K). Subsequent testing confirmed that Chinese output for Steps 4 and 5 can be fully generated within 16K tokens.

2.3 Preset Scanning Scenarios

ATM Scanner includes three preset scanning scenarios, each containing a target system description, known history, a known-vulnerability control group, and ATM analysis instructions. The complete input text for all three scenarios follows:

The design of all three scenarios follows ATM’s core principle: provide directional information about “where to look” (known vulnerabilities as control group + key design assumptions), not specific instructions about “what bug to find”. The Scanner’s AI analysis module autonomously executes the full pipeline of archaeology → seam marking → directed scanning → rule extraction on this basis.

03Test 1: Linux splice() Downstream Scan

The first preset scenario targets splice()’s zero-copy downstream consumers, with Dirty Pipe and Copy Fail information included as a control group.

The generated rules R1 (Parallel Write Channel Rule) and R2 (Assumption Stratigraphy Fault Rule) were both cross-validated against Linux kernel documentation and known CVEs. R2’s analysis of address_space‘s three semantic ownership changes (1991: VFS private → 2001: VFS/MM shared → 2007: MM can proactively revoke mappings) precisely matched kernel development history.

04Test 2: TCP Protocol Stack Seam Scan

The second preset scenario targets the Linux TCP protocol stack, spanning 44 years of design generations from RFC 793 (1973) to BBR (2016). This was the richest output of the three tests.

4.1 Archaeological Analysis Output Assessment

ATM Scanner identified five key design generations, with a fully verifiable timeline. Particularly noteworthy judgments include: TSO decoupled sequence number advancement from “physical byte transmission” to “kernel abstract computation” — an insight that directly determined the quality of subsequent seam marking; BBR’s RTprop measurement assumption (path transparency) is structurally absent in the modern middlebox ecosystem.

4.2 Seam Marking and Directed Scanning

Opus 4.6’s complete output flagged 5 high-risk seams and 4 medium-risk seams, and discovered two coupling chains between seams:

4.3 Seam Novelty Verification

A public literature search was conducted for each seam flagged by ATM Scanner to determine whether it represents a known vulnerability, a known but unexploited design flaw, or an entirely new unexplored area.

Verification conclusion: of the five high-risk seams, three (S1, S2, S5) are original findings by ATM Scanner with no corresponding CVE or security research in the public literature; one (S3) is known but reframed from a security perspective; one (S4) is partially known. These results demonstrate that ATM can systematically flag areas where traditional security auditing and fuzz testing are structurally blind.

4.4 Key Finding: Seam Coupling Chains

During scanning, Opus discovered a structural risk not anticipated in the original marking — the five high-risk seams exhibit two coupling chains, where a single triggering event can simultaneously activate multiple seams:

This finding embodies ATM’s core value: single-seam analysis underestimates systemic risk — only by tracing conflict propagation paths at the assumption layer through abductive reasoning can the amplification effects between seams be discovered.

05Test 3: VFS × MM Inter-Layer Seam Scan

The third preset scenario targets the interaction between VFS (Virtual File System) and the Memory Management subsystem, spanning 33 years of design generations from Linux VFS’s initial 1991 design to the 2023 introduction of the folio API. This scenario directly corresponds to the Dirty Cow and Copy Fail vulnerability family.

5.1 Archaeological Analysis: The Assumption Chasm Across Five Generations

ATM Scanner identified a continuously widening “assumption chasm” between VFS and MM: the VFS operation layer consistently assumes that target pages are in an exclusively writable state between write_begin and write_end; the MM layer, under multicore optimization pressure, has gradually evolved so that pages can be concurrently referenced by multiple paths, with ownership dynamically negotiated through a combination of reference counts and locks. These two assumptions are mutually exclusive by design, yet no refactoring has ever explicitly addressed this contradiction.

Five key generational crossings were precisely identified: Generation 1 (1991: buffer/page separation) → Generation 2 (2001: page cache unification, ownership separation broken) → Generation 3 (2004–2008: NUMA/multicore expansion, reference count intermediate states emerge) → Generation 4 (2013: THP compound pages, granularity semantic split) → Generation 5 (2020: folio introduction, attempted fix but created dual-track coexistence).

5.2 Seam Marking and Directed Scanning

5.3 SEAM-03: The Most Dangerous Structural Deficiency

SEAM-03 was flagged at 9/10 maximum risk because it does not require a race condition to trigger — unique among all seams across the three tests. The specific mechanism is as follows:

The Scanner provided precise scan coordinates: the coexisting paths of filemap_get_folio() and find_get_page() in mm/filemap.c, the folio migration state in ext4_write_begin() in fs/ext4/inode.c, and the diverging entry points of lock_page() and folio_lock() in include/linux/pagemap.h.

5.4 Generative Rules

The VFS × MM scan extracted 4 generative rules: R1 (Exclusive Assumption Illusion), R2 (Patch Band-Aid Recurrence), R3 (Gradual Migration Dual-Track Window), R4 (Measurement Granularity Split). Among these, R3 and TCP scan R4 (Dual-Track State Machine Regression) constitute the same meta-pattern across subsystems — different inputs ultimately point to the same underlying vulnerability generation mechanism, validating the convergence of generative rules.

06Prospective Prediction Validation: SEAM-03 Precisely Hit by Real CVEs

After completing the VFS × MM scan, we conducted a public literature search for all seams. The result yielded the strongest validation evidence for ATM to date: SEAM-03 (folio dual-track coexistence seam) was precisely hit by at least three real CVEs.

The Red Hat security advisory for CVE-2026-23097 is particularly critical — it explicitly states that the root cause is “incorrect lock ordering between folio_lock and i_mmap_rwsem”, which precisely corresponds to the core conflict flagged by ATM Scanner’s SEAM-03: “the new folio path assumes folio_lock() is the sole arbiter of ownership; the legacy page path assumes lock_page() locks independently at base page granularity.”

6.1 Complete Seam Novelty Verification Summary

Aggregating all results from the three scans, we conducted a public literature verification for each high-risk seam:

Of 9 high-risk seams: 4 are ATM original findings with no corresponding CVE (S1, S2, S5, SEAM-04); 1 is an ATM original finding subsequently validated by real CVEs (SEAM-03); 2 align with known research directions but with a novel perspective (S3, S4); 2 are rediscoveries of known vulnerability root causes (SEAM-01, SEAM-02).

07Sonnet 4 vs. Opus 4.6 Comparative Analysis

On the TCP protocol stack scenario, the same input was processed by both Sonnet 4 and Opus 4.6, producing a meaningful controlled comparison.

The most significant gap was in the depth of directed scanning (Step 4). Taking S1 as an example, Opus precisely located the short-circuit evaluation in tcp_validate_incoming() — if (tp->rx_opt.saw_tstamp && — when saw_tstamp is zero, the entire PAWS check is skipped, and the system silently degrades to 1974-era bare sequence number validation. Sonnet’s analysis reached only the conceptual level of “PAWS may fail” without touching the specific failure mechanism.

08Generative Rules Summary

The three tests collectively extracted 14 generative rules — 6 from the TCP protocol stack (Opus 4.6), 4 from splice() downstream, and 4 from VFS × MM. The common characteristic of these rules is: specific enough to guide directed scanning, yet abstract enough to be reusable across systems. The following lists representative core rules across scanning scenarios:

Cross-scenario rule convergence is particularly noteworthy: TCP-R4 (dual-track state machine regression) and VFS-R3 (gradual migration dual-track window) are different instantiations of the same meta-pattern; TCP-R2 (patch truncation semantic divergence) and VFS-R2 (patch band-aid recurrence) share the core structure of “fix introduces new seam.” This convergence indicates that vulnerability generation mechanisms have structural similarity across codebases — ATM’s core thesis receives empirical support here.

09ATM Efficiency Assessment

Opus 4.6’s output provided a quantitative efficiency assessment of ATM:

10Copy Fail: ATM Retrospective Validation

As an additional validation step, we applied the ATM five-step method to perform a post-hoc retrospective analysis of CVE-2026-31431 (Copy Fail), testing whether ATM could “derive Copy Fail from Dirty Pipe.”

10.1 Generative Rules Extracted from Dirty Pipe

Dirty Pipe (CVE-2022-0847) had already proven in 2022 that splice()’s zero-copy assumption was fragile. If someone had extracted the generative rule at that time — “On which paths is splice()’s zero-copy page reference assumed to be read-only?” — and scanned all downstream splice() consumers, Copy Fail’s habitat could have been located in 2022.

10.2 Structural Isomorphism with the Actual Discovery Process

The discovery process disclosed in Xint’s official blog is structurally isomorphic to ATM:

This is not post-hoc fitting — Xint’s workflow precisely corresponds to ATM’s Step 3 (human marks seams) → Step 4 (AI directed scan). Copy Fail’s discovery was an “unconscious practical validation” of ATM methodology.

11Single-Scan Error Rate Analysis

ATM Scanner’s foundation is a large language model (LLM) with approximately 5% sampling variability — errors in a single scan are inevitable and normal. An AI scanning tool that claims zero errors would be untrustworthy. Transparently reporting error rates is critical data for evaluating methodology effectiveness.

11.1 Confirmed Errors

11.2 Error Rate Statistics

11.3 Nature of Errors and Countermeasures

The above errors reveal an important capability stratification: ATM Scanner’s directional judgment capability (“where to look”) is significantly superior to its precise factual inference capability (“what exactly is there”). This aligns with ATM’s design objective — ATM’s value lies in narrowing the search space, not in replacing human auditing. If the direction is right, insufficient precision can be compensated by subsequent targeted human audit; if the direction is wrong, no amount of precision is useful.

Internal contradictions in numerical calculations (the same physical quantity appearing with different values in different steps) are a known LLM weakness. Two countermeasures exist: first, repeated scanning — executing multiple independent scans on the same target, taking the intersection as high-confidence results and flagging differences for human review; second, a numerical verification pipeline — adding a post-processing step to the Scanner that automatically verifies computationally deterministic physical quantities (e.g., sequence number wrap-around time = 2³² ÷ link byte rate).

11.4 Inherent Risk of LLM-Driven Security Scanning: The Hidden Error Problem

This experiment revealed a fundamental problem transcending ATM itself, pertaining to the entire field of AI-assisted security scanning: LLM-generated errors are hidden — they are formally indistinguishable from correct output.

When a human security researcher types “word scan” in conversation (meaning “single scan”), the error is immediately visible — the author sees it, the collaborator sees it, both laugh, correct it, and continue. A human typo has a definite physical event (finger hitting the wrong key); the lifecycle of the error from generation to detection to correction is transparent to all participants. This is an explicit error.

But when Opus 4.6 outputs “sequence number wraps around in ~14 seconds on a 40Gbps link” — this value, off by 16×, has no signal telling anyone it is wrong at the moment of generation. The model itself has no confidence markers; the output interface does not flag it in red; no confidence score is attached to the statement. It looks exactly like the perfectly correct sentences in the same paragraph, using the same font, the same tone, the same assertive certainty. The error actually occurred in GPU matrix multiplication — some attention head’s weight distribution gave “14” a higher generation probability than the correct “0.86” — but this process is completely unobservable to the outside. This is a hidden error.

11.5 Architectural Implications for AI-Assisted Security Scanning Tools

The hidden error problem imposes a fundamental architectural requirement on all LLM API-based security scanning tools: output cannot be treated as conclusions — only as candidates. Specifically:

First, human-machine collaboration is not optional but mandatory. ATM’s correct architecture is the three-stage workflow of “human sets direction → AI performs search → human validates results.” The AI output in the middle stage must be fact-checked by humans before it can be trusted. The actual discovery process of Copy Fail also validates this — the Xint team’s workflow was “human researcher provides direction → AI directed scan → human confirms results.”

Second, repeated scanning is the engineering countermeasure against hidden errors. LLM sampling variability (temperature) means that multiple scans on the same input produce different outputs. Erroneous output does not stably reproduce across multiple scans (because it is a stochastic artifact of probabilistic sampling), while correct directional judgments converge across multiple scans. By taking the intersection of multiple scans, hidden errors can be systematically filtered — analogous to noise averaging in signal processing.

Third, numerical output must have an independent verification pipeline. For computationally deterministic physical quantities (sequence number wrap-around times, memory address offsets, version numbers, etc.), an independent computational verification should be introduced in the Scanner’s post-processing stage — not relying on the LLM’s own reasoning, but using deterministic code to recalculate and compare against LLM output. This can raise the detection rate of numerical hidden errors to near 100%.

Fourth, an “AI scan result confidence classification” standard must be established. Currently, ATM Scanner’s output contains directional judgments (seam markers), mechanism inferences (attack primitives), and precise facts (numerical values/version numbers) with different reliability levels, but the output format makes no distinction. Future tools should attach different confidence tags to each output category, helping human reviewers quickly locate content requiring focused verification.

12Limitations and Future Work

12.1 Limitations

Prospective validation remains partial. SEAM-03 was validated by CVE-2025-37868 and CVE-2026-23097, proving ATM has prospective prediction capability. But this validation currently covers only 1 seam (out of 9 high-risk seams). The remaining 4 original findings (S1, S2, S5, SEAM-04) remain in “flagged but unvalidated” status, requiring subsequent real exploitation verification or confirmation by independent security researchers.

Limited sample size. Current validation is based on three subsystems of a single operating system (Linux kernel). ATM’s universality claim requires testing on more unrelated systems — e.g., database engines, browser rendering engines, distributed consensus protocols.

Artifact sandbox limitations. During testing, Step 5’s security analysis content triggered the Claude.ai Artifact previewer’s content security policy, blocking results in the preview panel. This does not affect conversation-area output but limits the user experience of ATM Scanner as a standalone application.

12.2 Future Work

S1 empirical validation. S1 (PAWS Silent Degradation), flagged in the TCP protocol stack scan, is the seam with the highest empirical validation value — reproducing “timestamp stripped, then sequence number validation fails in wrap-around scenario” on a 10GbE+ test environment. If successful, this would constitute ATM’s second prospective prediction validation.

ATM Scanner independence. Extract the Scanner from the Artifact sandbox and release it as a CLI tool or standalone web application, eliminating content filtering restrictions.

Rule library continuous accumulation. Each successful seam scan should add newly discovered generative rules to the library, building a searchable “vulnerability genome database.” The current 14 rules are just the beginning — as more systems are scanned, the rule library’s cross-system predictive power should continuously increase.

Cross-ecosystem blind testing. Select an open-source project ATM Scanner has never encountered (e.g., PostgreSQL or a Chromium submodule), conduct a full five-step scan, and have results verified by independent security researchers.

13Conclusion

ATM methodology has progressed from paper to code, from code to empirical testing, and from testing to prospective prediction validation. All three tests produced meaningful results, with core findings summarizable in five points:

First, ATM Scanner has prospective prediction capability. SEAM-03 (folio dual-track coexistence seam, 9/10) flagged in the VFS × MM scan was precisely hit by CVE-2025-37868 and CVE-2026-23097 in post-hoc search. The Scanner predicted their habitat purely through abductive reasoning with no knowledge of these CVEs. This is the key evidence for ATM’s upgrade from “post-hoc explanation” to “prospective prediction.”

Second, ATM Scanner is not a toy. Three scans collectively flagged 9 high-risk seams, of which 4 are original findings with no corresponding CVE (S1, S2, S5, SEAM-04), 1 was precisely validated by real CVEs (SEAM-03), 2 independently aligned with known research directions (S3, S4), and 2 are rediscoveries of known vulnerability root causes (SEAM-01, SEAM-02).

Third, generative rules exhibit cross-system convergence. Three scans produced 14 generative rules in total, with rules from different subsystems showing clear convergence — TCP’s R4 (dual-track state machine regression) and VFS/MM’s R3 (gradual migration dual-track window) are different instantiations of the same meta-pattern, indicating that vulnerability generation mechanisms have structural similarity across codebases.

Fourth, ATM effectiveness is positively correlated with model reasoning capability. Opus 4.6 produced twice as many generative rules as Sonnet 4, and its unique R6 (chain activation) revealed the structural blind spot of single-seam analysis. The methodology’s ceiling is determined by model capability, not by the methodology itself — meaning ATM output quality will scale in step with ongoing improvements in base models.

Fifth, LLM-driven security scanning carries an ineliminable hidden error risk. This experiment documented a ~6% mechanism misattribution rate and ~10% numerical deviation rate, with these errors formally indistinguishable from correct output — the model cannot self-check, the output interface cannot flag them, and monitoring systems cannot capture them. This is an inherent risk characteristic of all LLM API-based security scanning tools, dictating that ATM must be a “human-machine collaboration” architecture rather than an “AI autonomous operation” architecture. The correct positioning of AI security scanning is an efficiency amplifier, not a judgment replacer.

14References

ATM Architecture Demo Test · V2

이조글로벌인공지능연구소 · LEECHO Global AI Research Lab

& Opus 4.6 · Anthropic


May 1, 2026