KB-18FE

FIX7 Recheck-9 V3 — R9-B6 Black-Box CLI Oracle Master Report (2026-06-10)

10 min read Revision 1
tool-kiem-thufix7recheck-9packet-v3r9-b6blackbox-cli-oraclearticle-142026-06-10

FIX7 Recheck-9 V3 — R9-B6 Black-Box CLI Oracle — Master Report

  • Date: 2026-06-10 · Lane: Approval Lane Macro FIX7_RECHECK9_V3_R9_B6_BLACKBOX_CLI_ORACLE_PATCH
  • Final status: FIX7_RECHECK9_V3_BLACKBOX_CLI_ORACLE_SELF_CODEX_PASS
  • Production mutation: NO · Codex consulted: NO · T2 dependency: NO
  • Authority: provisional-non-authority; evidence-only; decision_effect=NONE; may_gate=false. Nothing here is sealed.

1. Readback table (macro §0)

item content
Codex V2 finding R9-V2-B6: negative tests never execute the real CLI; manifest_tool.run_negative_tests() calls canon.produce() in-process and synthesizes cli_exit_contract = 0 if corpus_ok … else 4. Codex mutated the CLI error branch sys.exit(4)→sys.exit(0), regenerated expected output/manifest/HASH with the packet's own tools, and full RERUN passed 22/22 while missing-doc actual CLI exited 0.
Root cause Oracle laundering: SUT-derived state was used as both the test subject and the expected value; the load-bearing exit-code claim was an inference, not an observation; --emit would re-derive "truth" from whatever the SUT did.
Why V2 selfcheck missed it The V2 adversarial suite (T2) also asserted produce() in-process state, and --verify deep-diffed recomputed-vs-disk values that were BOTH derived the same synthesized way — self-consistent under any SUT mutation. No test path ever observed a process/SystemExit exit code.
Exact files affected manifest_tool.py (run_negative_tests, compute_authority, cmd_verify), adversarial_suite.py (T2), RERUN.sh (no negative CLI gate), manifest.json (cli_exit_contract, exit_code_contract fields).
Required black-box check Execute the ACTUAL canonicalizer CLI for every negative corpus case and both positive controls; compare OBSERVED exit + stdout markers + digest suppression + artifact absence against a STATIC spec-derived oracle; make --emit/--verify fail closed on violation; add a fail-open regression replaying the exact Codex mutation.
Acceptance test (a) canonical packet: full RERUN exit 0 with negative CLI exits OBSERVED == 4; (b) Codex mutation applied to a copy: --emit exit ≠ 0 with ORACLE_VIOLATION, full RERUN exit ≠ 0 BEFORE any PASS, black-box + adversarial suites exit ≠ 0.
Expected failure before fix Reproduced first-hand on Packet V2: BYPASS_RERUN_EXIT=0, RERUN_RESULT: PASS, ADVERSARIAL_SUITE: PASS (22/22), BYPASS_MISSING_DOC_CLI_EXIT=0.
Expected pass after fix Achieved on Packet V3 (see §3/§4).

2. What V3 changes (SUT / ORACLE / VERIFIER separation — R9-B6.1)

  • SUT = the actual CLI files executed as __main__: canonicalizer (--produce, --selftest) and, for verifier-lane cases, manifest_tool.py itself. Two execution levels: true OS-process from bash (RERUN gate 6, $? observed) and manifest_tool.run_cli (runpy harness: real argv, fresh module run, OBSERVED SystemExit code, captured stdout/stderr, process semantics mirrored — string-code SystemExit → message + exit 1, unhandled exception → 1). No child-process spawning; the forbidden-scope policy is intact (Codex explicitly allowed a runpy/SystemExit harness).
  • ORACLE = manifest_tool.CLI_ORACLE: STATIC spec pin from the SSOT contract text (produce ok→0, corpus error→4, selftest pass→0/fail→1, suppression marker, suppressed token). A verifier constant; never regenerated from SUT output. evidence/*-expected-output.txt are explicitly reclassified as SUT-derived VALUE-CONSISTENCY pins, NOT behavioral oracles (recorded in manifest explanatory.expected_output_pins_classification).
  • VERIFIER = blackbox_negative_suite.py (10 observed-behavior checks), failopen_regression.py (6 checks replaying the Codex attack), run_negative_tests/compute_authority (raise ORACLE_VIOLATION on any observed-vs-oracle mismatch at BOTH --emit and --verify), RERUN gates 6/7/8/11.
  • Synthesized fields REMOVED: cli_exit_contract and exit_code_contract no longer exist; replaced by cli_exit_observed + cli_exit_expected + cli_exit_matches_oracle + evidence_class: EXECUTED_CLI_BLACKBOX. RERUN gate 11 hard-fails if cli_exit_contract ever reappears in manifest.json.

3. Canonical Packet V3 evidence (all OBSERVED, raw logs in packet logs/ + rerun-out/)

  • bash RERUN.sh → exit 0, RERUN_RESULT: PASS (all 13 gates re-executed and enforced in this invocation); gate 6 lines: negative CLI case 'missing'|'extra'|'invalid'|'absentdir': OBSERVED exit 4 + suppression markers + zero digest leak (raw per-case CLI stdout in rerun-out/neg-*.log).
  • blackbox_negative_suite.py → exit 0, PASS (10/10 observed-behavior checks … none inferred).
  • failopen_regression.py → exit 0, PASS (6/6) incl. M2 proof the mutated SUT genuinely exits 0 and M3–M5 proof every gate rejects it.
  • manifest_tool.py --verify → exit 0: ENTIRE authority recomputed (18 top fields, 27 artifact hashes, selftest 45/45, 9 negative tests, 6 REAL CLI executions observed against the static oracle).
  • adversarial_suite.py → exit 0, PASS (25/25) (V2's 22 + T2d observed-CLI-exit + T13a/T13b fail-open emit/verify).
  • Packet completeness 33 required files; HASH_MANIFEST 32 entries; V3 tree hash b95df0a5d2f41f80bea0cef8621c1f8bb0f6b49a40175116418494ed4141ca6d (32 tracked files).
  • KB-native (R9-B4 preserved): kb_fetch_reconstruct.py fresh-fetch → RECONSTRUCTION: OK (32 files…); bash RERUN.sh on the reconstructed tree → exit 0 PASS; reconstructed tree hash identical b95df0a5….
  • SSOT unchanged: rev3 candidate 49c386a9b9666c09786fc4f89bc79776b6046eaee6f4da6d8537d2c753b734d0; frozen membership f2bda8ef…fe251 reproduced. The SUT was NOT modified by this lane — only the proof toolchain.

4. The Codex V2 attack: reproduced, then defeated (R9-B6.3)

Before fix (Packet V2 copy, exact Codex sequence — mutate sys.exit(4)→sys.exit(0) in SSOT fence + .py + extended, regenerate expected outputs + manifest + HASH with V2 tools): BYPASS_RERUN_EXIT=0 · RERUN_RESULT: PASS · ADVERSARIAL_SUITE: PASS (22/22) · missing-doc actual CLI BYPASS_MISSING_DOC_CLI_EXIT=0 while stdout prose printed "-> exit 4". Defect class confirmed first-hand.

After fix (same mutation + same laundering sequence on a Packet V3 copy): V3_BYPASS_EMIT_EXIT=1 printing ORACLE_VIOLATION … 'cli_exit_observed': 0, 'cli_exit_expected': 4 · --emit-hash-manifest still hashes bytes (by design: transport integrity, not behavior) · bash RERUN.shV3_BYPASS_RERUN_EXIT=1, aborting at gate 6 with NEGATIVE CLI CASE 'missing': OBSERVED exit 0 != 4 (oracle) — FAIL-OPEN + RERUN_RESULT: FAIL (black-box negative CLI gate) BEFORE any PASS · adversarial_suite.py exit 1 · mutated missing-doc CLI observed exit 0 (the fail-open exists and is detected). The same regression runs live inside canonical RERUN gate 8 on an OS-temp copy (canonical tree byte-identical before/after, asserted by M6).

5. Verdict summary

lane verdict
Codex V2 failure reproduction REPRODUCED first-hand on V2 bytes (not merely mapped)
Black-box CLI negative suite PASS 10/10, all exits OBSERVED
Fail-open regression PASS 6/6; both standalone and live in RERUN gate 8
Manifest laundering prevention PROVEN: --emit exit 1 ORACLE_VIOLATION on fail-open SUT
RERUN strictness V3 13 gates live; negative CLI executed at OS-process level; inferred-evidence fields rejected
Adjacent self-referential scan 9 surfaces checked; 4 fixed, 4 documented-by-design, 1 honest structural limit (verifier self-integrity → Codex independent rerun backstop)
Packet V3 KB-native PASS: 14 files published byte-verified; fresh-fetch reconstruction + RERUN PASS; tree b95df0a5…
Article 13 PASS: KB packet root is the authority surface; fresh fetch, not local mirror; no production/PG/Directus/registry-pivot mutation
Article 14 PASS for this lane: every executable claim carries actual command + OBSERVED exit + raw log + hash; no inferred exit anywhere in the proof chain

6. Remaining blockers (authority/Codex only — no engineering blocker remains in this lane)

  1. N7 envelope_manifest — sealed approval-event inputs (Codex/owner). 2. N8 detached seal — Codex-authored. 3. P7 — Codex re-seal over SSOT rev3 candidate 49c386a9…. 4. Owner standing do-not-approve. 5. R9-B5 residual — no server-side governed digest endpoint (MCP-byte proof stands; Codex V2 accepted this level). N/A-with-rationale: duplicate-active-doc ON DISK not representable on a case-insensitive FS — covered by the executed duplicate-listing validator (blackbox N5′ + SSOT selftest fixtures); recorded as N/A, NOT marked PASS.

7. Companion documents

Reproduction report, black-box suite report, fail-open regression report, laundering-prevention report, adjacent-scan report (same folder, same date); handoff + checkpoint under checkpoints/; current-state under knowledge/current-state/reports/. Packet root: knowledge/dev/laws/tool-kiem-thu/packets/fix7-codex-recheck-9-2026-06-10/.

NEXT: route Packet V3 → fresh Codex Recheck-9 rerun (V3). No FIX7 implementation, REAL_RUN, QT001, permit, activation, repoint, cutover, registries-pivot, or auto-birth repair was performed or authorized.

Back to Knowledge Hub knowledge/dev/laws/tool-kiem-thu/reports/fix7-recheck9-v3-r9-b6-blackbox-cli-oracle-master-report-2026-06-10.md