Skip to the matrix

Attack lab

Simulatedunit / attack-harness/1 / oracle-v1

Every attack. Every baseline.

Attack lattice: 24 attacks against 4 targets, 4 run. 35 cells are losses.
Target01ATK-0102ATK-0203ATK-0304ATK-0405ATK-0506ATK-0607ATK-0708ATK-0809ATK-0910ATK-1011ATK-1112ATK-1213ATK-1314ATK-1415ATK-1516ATK-1617ATK-1718ATK-1819ATK-1920ATK-2021ATK-2122ATK-2223ATK-2324ATK-24
Symbolon (unit mode: real kernel-core pipeline)
B0 (circlefin/arc-escrow release path, pinned, edges stubbed)
B1 (naive release-on-model-output, ours)
B2 (reasonable hardening, ours)
  • Blocked
  • Escalated
  • Held
  • Loss
  • No effect
  • Not applicable
  • Not run
  • Not built yet
  1. Symbolon (unit mode: real kernel-core pipeline)

    0 of 24

    No loss on 24 applicable attacks.

    Not applicable
    0
    Trials
    213
    Oracle-scored
    0 lost / 19
    Signal-scored
    0 lost / 5
    Loss rate, as provided
    0/213 loss trials, 95 % upper bound 0.0141 (rule-of-three)
  2. B0 (circlefin/arc-escrow release path, pinned, edges stubbed)

    4 of 20

    Loss on 4 of 20 applicable attacks.

    Not applicable
    4
    Trials
    168
    Oracle-scored
    4 lost / 17
    Signal-scored
    0 lost / 3
    Loss rate, as provided
    24/168 loss trials, 95 % upper bound 0.1950 (clopper-pearson)
  3. B1 (naive release-on-model-output, ours)

    19 of 20

    Loss on 19 of 20 applicable attacks.

    Not applicable
    4
    Trials
    168
    Oracle-scored
    16 lost / 17
    Signal-scored
    3 lost / 3
    Loss rate, as provided
    92/168 loss trials, 95 % upper bound 0.6127 (clopper-pearson)
  4. B2 (reasonable hardening, ours)

    12 of 20

    Loss on 12 of 20 applicable attacks.

    Not applicable
    4
    Trials
    168
    Oracle-scored
    9 lost / 17
    Signal-scored
    3 lost / 3
    Loss rate, as provided
    68/168 loss trials, 95 % upper bound 0.4709 (clopper-pearson)

Not-applicable attacks stay in the table with their reason and are excluded from the denominators. Signal-scored rows are tagged: they are not oracle-scored.

Attack lattice: 24 attacks against 4 targets, 4 run. 35 cells are losses.
AttackSymbolon (unit mode: real kernel-core pipeline)B0 (circlefin/arc-escrow release path, pinned, edges stubbed)B1 (naive release-on-model-output, ours)B2 (reasonable hardening, ours)
BlockedSignalNo effectSignalLoss8.00 USDCSignalLossSignal
BlockedNo effectLoss8.00 USDCBlocked
EscalatedLossLossLoss
BlockedBlockedLoss21,964.00 USDCBlocked
BlockedBlockedBlockedBlocked
BlockedNo effectLoss2.00 USDCLoss2.00 USDC
BlockedBlockedLoss8.00 USDCLoss8.00 USDC
BlockedNo effectLoss8.00 USDCBlocked
Not runNo effectSignalLossSignalLossSignal
BlockedBlockedLoss648.00 USDCBlocked
BlockedNo effectLoss8.00 USDCBlocked
BlockedNo effectLoss24.01 USDCLoss16.01 USDC
BlockedBlockedLoss8.00 USDCBlocked
BlockedNo effectLoss86.10 USDCLoss82.90 USDC
HeldLossLossLoss
BlockedLossLoss16.00 USDCLoss16.00 USDC
Not runNot applicableNot applicableNot applicable
BlockedNo effectLoss152.00 USDCBlocked
BlockedSignalNo effectSignalLossSignalLossSignal
BlockedNo effectLoss32.00 USDCLoss32.00 USDC
BlockedLoss8.00 USDCLoss30.00 USDCLoss10.00 USDC
EscalatedNot applicableNot applicableNot applicable
BlockedSignalNot applicableNot applicableNot applicable
EscalatedSignalNot applicableNot applicableNot applicable

Hold-out

Not run. the hold-out set is written by ATTACK-ADV outside this repository and run in WP-36 (H1 at G3, H2 at G5); this harness never reads it

Honest corpus

  • symbolon12 released of 12; 0 wrongful rejectsupper bound as provided: 0.242 (Wilson 95 % upper bound on wrongful rejects)
  • b012 released of 12; 0 wrongful rejectsupper bound as provided: 0.242 (Wilson 95 % upper bound on wrongful rejects)
  • b112 released of 12; 0 wrongful rejectsupper bound as provided: 0.242 (Wilson 95 % upper bound on wrongful rejects)
  • b212 released of 12; 0 wrongful rejectsupper bound as provided: 0.242 (Wilson 95 % upper bound on wrongful rejects)

Reproducibility

Git sha
37da234b8dd5a990fecfb91fe6517e2e1c9acbcd
Seed
base
Models
none: no model was called (reference extraction reads; the B0/B1/B2 validator is a deterministic SIMULATED stand-in)
Fixture manifest
0x9a98e37726ef…5a0bbe1fa7
No model was called: simulated

Known limits

  • L-01Engagements are the fixture templates activated with placeholder signatures and a placeholder registration (SIMULATED); no key signed anything and nothing was registered on a vault.
  • L-02Extraction reads are the fixture generator's REFERENCE reads, not model outputs; no model was called. B1's validator is a deterministic stand-in for an LLM validator (SIMULATED). Nothing in this report is labelled RECORDED because nothing was recorded.
  • L-03ATK-01, ATK-09, ATK-19, ATK-23 and ATK-24 have no oracle-expressible loss (docs/QUESTIONS WP-22 item 2). Their results are signal-scored or NOT_RUN and are counted separately from oracle-scored attacks.
  • L-04ATK-17 (contract calls) and the fault steps (reconciler, K-15) need e2e-local; the Symbolon target reports them NOT_RUN with the reason. ATK-22 (digest), ATK-23 (WYSIWYS comparator) and ATK-24 (explainer) run against kernel-core and packages/receipts; whether the web screen really disables Sign is WP-19/SEC-13, not measured here.
  • L-05Trials are extraction runs of the committed fixtures. Runs of one fixture share a payee, an amount and a document and are not independent, so the upper bounds are optimistic; the hold-out set (WP-36) is the independent evidence.
  • L-06Decisions accumulate across the steps of a fixture (docs/QUESTIONS WP-22 item 9). K-04 therefore sees earlier steps of the same fixture as near-duplicates, replays or as a milestone already paid (and K-04 rejects a later instalment of a partialAllowed milestone); the later checks the attack was designed for are then not exercised in that run. See docs/QUESTIONS WP-23 (re-baseline).
  • L-07B0 is the vendored validate-work route of circlefin/arc-escrow EXECUTED with its network edges stubbed (vendor/arc-escrow/VENDOR.md); its model call is a deterministic SIMULATED validator, and arc-escrow validates an image deliverable against agreement tasks, not an invoice, so our documents are mapped onto its job/deliverable API (applicability.json carries the mapping and a reason per row). Where B0 holds by design (payee and amount are fixed at agreement creation) it is reported as measured, not dropped.
  • L-08B2 is our own 'reasonable hardening' (schema check, allowlist, per-tx and daily caps, exact duplicate-number check) on the B1 pattern; it is not Circle's code and not the only reasonable choice. It is published in src/harness/b2.ts so others can improve it.
  • L-09B1 pays the face amount of the document and has no terms logic, so it overpays the early-pay discount on honest invoices; that is part of the pattern, and it is reported, not hidden.
  • L-10Every scenario instance is prefunded with the same 1,000,000 USDC; a transfer above the remaining balance reverts and moves no money, so losses are bounded by it.