R1a-R2a Runner / Cron / Log Root-Cause Study — Execution Report (2026-06-18)
R1a-R2a Runner / Cron / Log Root-Cause Study — Execution Report
Date: 2026-06-18 · Workstream: R1a ∥ R2a (read-only runner/cron/log root-cause study, after accepted R1/R2 read-only scoping baseline + Codex PASS_WITH_CAVEATS) · Revision: rev1
Class: execution report / read-only root-cause / Owner-decision-prep
READ-ONLY · NON-ENACTING · NON-AUTHORIZING · NOT remediation · NOT technical design · NOT implementation · NO blocker resolved · NO restart · NO job/DOT/KG/birth/certify execution.
0. Status
STATUS: PASS — both R1a and R2a root causes are identified with direct, first-hand read-only evidence at the runner / scheduler / config layer that R1 and R2 left as their PARTIAL gaps. This run discovered the missing runner/cron/log layer for both packages; it did not repair, restart, execute, certify, or backfill anything.
- R1a: A KG runner exists and is healthy (
incomex-agent-api-executor:8090/dispatch), but KG real-run is deliberately fail-closed NO_GO (v_dotkg_realrun_preflight = REALRUN_BLOCKED_MULTI_GATE, 5 owner-gates BLOCK); only 1/36 KG DOTs is contracted (EXPLAIN pilot, DRY_RUN); the 2199 edges were seeded (LEGACY|S167H) + Directus-synced with 0 provenance and the provenance-tag DOT never ran → no provenance source-of-truth. - R2a: The 2026-03-21 certification was a one-shot, operator-run S157-A bootstrap (
dot-birth-backfill+s157bseed via SSH+docker exec, stamping inspect_*+certified in the INSERT); it never recurred because the inspect→certify producer was never operationalized — inspector DOTs are unwired stubs, no pg_cron, host0 6 * * *belongs todot-nrm-lifecycle; the auto-certify consumer is healthy but starved.
PASS = engineering root cause for both packages. It is not an Owner authorization to remediate. Engineering verification ≠ Authority approval. Every blocker remains OPEN; default disposition HOLD.
1. Files created
Exactly three new reports (no source/law/draft/note/prior-report patched):
| File | Revision | content_length | Status |
|---|---|---|---|
reports/r1a-kg-runner-log-provenance-source-root-cause-2026-06-18.md |
1 | 27168 | created · re-read · verified |
reports/r2a-birth-inspection-runner-cron-log-root-cause-2026-06-18.md |
1 | 23425 | created · re-read · verified |
reports/r1a-r2a-runner-cron-log-root-cause-execution-report-2026-06-18.md |
1 | 14799 | created · re-read · verified |
2. Sources read
Report baseline (full, first-hand): R1 scope (rev1), R2 scope (rev1), R1-R2 parallel exec report (rev1), Phase-1 read-only runtime blocker verification (rev1), Codex review PASS_WITH_CAVEATS (reports/codex/…2026-06-18.md), Phase-1B carry. Governance: .claude/skills/incomex-rules.md (3 tuyên ngôn + non-mutation guard), project CLAUDE.md. R1/R2 anchors carried from the accepted baseline (Điều 39 law+note; Điều 4 birth note, L4 amendment, Điều 0-G birth-registry-law, law-04). Deployed producer source read first-hand from the synced local mirror: web-test/dot/bin/dot-inspect-pen, dot-birth-backfill, dot-birth-trigger-setup. No required source was unreadable.
Codex's recommended next macro ("a second read-only runner/cron/log root-cause macro first, split into R1 and R2 lanes") is exactly what this run performed.
3. Commands run
All read-only. SQL via query_pg (AST-validated, READ ONLY transaction, read-only role, statement_timeout 5s, LIMIT 500) against directus; Docker via list_docker/docker_logs (list/tail only); local Read/ls/find/wc. Session window 2026-06-17 23:49 → 2026-06-18 01:30 UTC. No write/DDL/DML/execution/restart was made or prepared; the only non-ok results were a denied docker_logs (executor not in allowlist) and an earlier-session pg_schema helper bug (not used this run) — neither a write.
| ID | Command (abbrev.) | Target | Read-only? | Exit | Pkg |
|---|---|---|---|---|---|
| D0 | list_docker (11 containers) |
VPS | yes | ok | both |
| Q1 | pg_extension (no pg_cron) |
directus | yes | ok (4) | both |
| Q2 | dot_tools columns |
directus | yes | ok (28) | both |
| Q3 | KG-DOT detail (5 codes) | directus | yes | ok (5) | R1a |
| Q4 | birth-DOT detail (%BIRTH%) |
directus | yes | ok (5) | R2a |
| Q5 | run/exec/job/queue table inventory | directus | yes | ok (68) | both |
| Q6 | dot_tools trigger_type distribution |
directus | yes | ok (7) | R1a |
| Q7 | dot_config columns |
directus | yes | ok (4) | both |
| Q8 | columns for 12 runner/queue/log tables | directus | yes | ok (154) | both |
| Q9 | dot_config runner/queue/kg/birth keys |
directus | yes | ok (19) | both |
| Q10 | birth_registry certified GROUP BY origin |
directus | yes | ok (3) | R2a |
| Q11 | v_process_discovery_runner_status |
directus | yes | ok (17) | both |
| Q12 | wf_host_crontab_snapshot (host cron) |
directus | yes | ok (54) | R2a |
| Q13 | queue_heartbeat |
directus | yes | ok (3) | both |
| Q14 | recency/count union (9 run/log tables) | directus | yes | ok (9) | both |
| Q15 | v_dotkg_realrun_preflight (gates) |
directus | yes | ok (10) | R1a |
| Q16 | pg_get_viewdef(v_dotkg_realrun_preflight) |
directus | yes | ok (1) | R1a |
| Q17 | dot_agent_api_contract |
directus | yes | ok (2) | R1a |
| Q18 | process_run_observation |
directus | yes | ok (6) | R1a |
| Q19 | dot_config process_dot_runtime.* |
directus | yes | ok (5) | both |
| Q20 | governance_object_ownership |
directus | yes | ok (0) | R1a |
| Q21 | event_outbox breakdown |
directus | yes | ok (8) | both |
| Q22 | pg_settings WHERE name LIKE 'app.%' |
directus | yes | ok (0) | R2a |
| Q23 | pg_db_role_setting |
directus | yes | ok (0) | R2a |
| Q24 | universal_edges GROUP BY _dot_origin,edge_type |
directus | yes | ok (5) | R1a |
| G1–G3 | docker_logs postgres/directus/agent-data (tail 5) |
VPS | yes | ok | both |
| G4 | docker_logs incomex-agent-api-executor |
VPS | yes | DENIED (allowlist) | R1a (gap) |
| E1–E8 | local date/ls/find/wc/Read deployed source |
local mirror | yes | ok | both |
Tool-boundary contingency recorded. No docker exec/docker inspect/docker compose/crontab -l/systemctl/SSH tool is available; read_file is allowlisted to /opt/incomex/docs, /opt/incomex/dot/specs, /var/log/nginx only. Host cron was inspected via the read-only DB-captured wf_host_crontab_snapshot; the executor's process logs and the transient-session GUC layer are honest EVIDENCE_GAPs (recorded, not inferred). They do not change either verdict because the DB-side dispatch ledger, preflight gates, runner-status views, host-crontab snapshot, and producer scripts are independently conclusive.
4. R1a completion check
- C3 container inventory: ✓ (D0 — runner
incomex-agent-api-executor:8090healthy). - C4 runner/cron/service: ✓ (Q5/Q8/Q9/Q11/Q13 — master switches off, queue idle since 2026-05-26, KG family
mixed_engine_partial_runner). - C5 logs: ✓ where available (G1–G3 liveness; executor log DENIED = recorded gap;
process_run_observation= 6 dry-run only). - C6 KG-runner code/config: ✓ (Q3/Q6/Q15/Q16/Q17/Q19/Q20 — preflight NO_GO, only EXPLAIN contracted, owner table empty).
- C8 R1a root-cause verdict: ✓ (REGISTERED_NOT_EXECUTED = governed fail-closed gating + no provenance SoT).
- KG questions answered: runner exists ✓ / not enabled for real-run ✓ / kg.* only EXPLAIN-routed in DRY_RUN ✓ / 0/36 cause = 5-gate NO_GO + no scheduler ✓ / provenance SoT absent ✓ / Qdrant = vector-search not provenance ✓ / pre-write-R1 requirements stated ✓.
R1a fully scoped at the runner/cron/config layer. PASS.
5. R2a completion check
- C3 container inventory: ✓ (D0 — no birth-inspection service; postgres trigger fabric + manual CLIs).
- C4 runner/cron/service: ✓ (Q11
dot:birth=engine_unclassified/requires_runner; Q13 no birth executor). - C5 logs: ✓ where available (cannot reach 2026-03-21 = recorded gap; reconstructed from Q10 provenance + script content, which agree).
- C7 birth inspect-producer code/config: ✓ (Q4 stubs;
dot-inspect-pen/dot-birth-backfillread first-hand; Q12 host cron; Q1 no pg_cron; Q22/Q23 GUC catalog). - C9 R2a root-cause verdict: ✓ (one-shot manual S157-A bootstrap; producer never operationalized).
- Birth questions answered: no actual runner ✓ / cron
0 6=dot-nrm-lifecyclenot birth-verify ✓ / 2026-03-21 batch = manual ✓ / never recurred because never wired ✓ / GUC persisted layer empty (no bypass), transient unreadable ✓ / pre-write-R2 requirements stated ✓.
R2a fully scoped at the runner/cron/config layer. PASS.
6. Forbidden-action check
| Forbidden action | Occurred? |
|---|---|
| write / DDL / DML (INSERT/UPDATE/DELETE/CREATE/ALTER/DROP/TRUNCATE/GRANT/REVOKE) | No — read-only role, READ ONLY txn; all SQL SELECT/catalog |
| restart / reload container or service | No |
| run worker / cron / job | No |
| trigger DOT / KG / birth / certify / promote / repair execution | No |
set inspect_pen/inspect_stamp/inspect_gate / certified=true |
No |
| backfill provenance / quarantine edges | No |
flip dot_config gate / assign owner / promote contract DRY_RUN→REAL_RUN |
No |
| write env/config files / patch source / patch prior KB report | No — only 3 new reports created |
| create current corpus | No |
| write technical design | No — design-direction items flagged gated, not designed |
| implement | No |
| resolve blocker | No — all OPEN |
| materialize KG/provenance/stamps/cell_id/dot_role/canonical | No |
| change authority order / v0.1 baseline / promote v0.2-hardening | No |
No NOT_SAFE_TO_TEST action was attempted. No forbidden mutation occurred → not FAIL. (The denied docker_logs was a read attempt, recorded as a gap.)
7. Findings summary
13 findings (7 R1a, 6 R2a; G-items are gaps). 8 HIGH, 1 MEDIUM, 4 INFO/LOW. No CRITICAL. No active mutation/bypass/execution observed.
| ID | Sev | Pkg | Summary | Blocks write-enabled remediation? |
|---|---|---|---|---|
| R1a-F1 | HIGH | R1a | KG real-run fail-closed NO_GO (v_dotkg_realrun_preflight, 5 BLOCK gates); 0 REAL_RUN |
Yes |
| R1a-F2 | HIGH | R1a | Only 1/36 KG DOTs contracted (EXPLAIN, DRY_RUN); provenance DOTs unrouted | Yes |
| R1a-F3 | HIGH | R1a | No provenance source-of-truth (edges seeded/Directus-synced, 0 provenance; tag-DOT never ran) | Yes |
| R1a-F4 | MED | R1a | DOT runtime disabled (process_dot_runtime.*=false) + queue idle since 2026-05-26 |
Yes |
| R1a-F5 | LOW/asset | R1a | Healthy KG runner endpoint + no-mutation boundary + 2 dry-run proofs (untested for real-run) | No |
| R1a-F6 | INFO | R1a | kg_quality_log=0 is downstream of never-dispatched quality DOTs; no pg_cron installed |
No |
| R1a-G1 | INFO(gap) | R1a | Executor docker_logs DENIED → process-level dispatch logs not inspectable |
No |
| R2a-F1 | HIGH | R2a | No standing inspection producer/runner; inspector DOTs are unwired stubs; producers are manual CLIs | Yes |
| R2a-F2 | HIGH | R2a | 2026-03-21 certification = one-shot manual S157-A bootstrap (not cron/runner/migration) | Yes |
| R2a-F3 | HIGH | R2a | Cron not wired: no pg_cron; host 0 6=dot-nrm-lifecycle; no birth cron/systemd entry |
Yes |
| R2a-F4 | HIGH | R2a | 1,211,557 uncertified births grow live; auto-certify consumer starved; outbox undrained | Yes |
| R2a-F5 | MED/INFO | R2a | GUC persisted layer empty → warn-mode default, no engaged bypass; transient layer unreadable (gap) | No |
| R2a-G1 | INFO(gap) | R2a | Container logs can't reach 2026-03-21; bin/env outside allowlist → reconstructed from provenance+script | No |
Consistency. Reconfirms and sharpens R1 (REGISTERED_NOT_EXECUTED), R2 (single 2026-03-21 batch / producer absent), and Phase-1B (Đ39 runtime-EMPTY; HOLD-2 PARTIAL). No contradiction. New precision: the KG "why-0/36" is a deliberate 5-gate fail-closed preflight + missing provenance SoT (not a broken runner); the birth "why-stalled" is a never-operationalized manual bootstrap (not a pipeline that broke).
8. Non-authorization
This report and its two siblings authorize nothing. No DDL/DML; no runtime write; no restart/reload; no worker/cron/job run; no KG/DOT/birth/certify/promote execution; no provenance backfill / edge quarantine; no inspect_*/certified set; no dot_config gate flip / owner assignment / contract promotion; no stamp/canonical/cell_id/dot_role materialization; no source/draft/note/prior-report patch; no current corpus; no draft adoption; no technical design; no implementation; no blocker resolved; no authority-order (CONS-004) change; no v0.1 baseline change; no v0.2-hardening promotion. R1 and R2 remain scoped read-only, not opened for remediation. Engineering/Codex PASS ≠ Owner authorization. CONS-002/003 + CELL-003/004/007 (and Điều 0-G source-recovery for R2) remain prerequisites to any R1/R2 materialization. Default disposition: HOLD.
9. Self-check
| Check | Result |
|---|---|
| SC1 Read R1/R2 baseline? | Yes (R1, R2, R1-R2 exec, Phase-1, Phase-1B) |
SC2 Read official Codex review (reports/codex/)? |
Yes (…2026-06-18.md, PASS_WITH_CAVEATS) |
| SC3 Inspected actual runtime runner/cron/log surfaces? | Yes (containers, dot_config gates, preflight view, runner-status view, queue heartbeat, host-crontab snapshot, run/log tables, producer scripts) |
| SC4 Used only read-only commands? | Yes (read-only role, READ ONLY txn; list/tail Docker; local reads) |
| SC5 Avoided all restarts/jobs/triggers? | Yes |
| SC6 Avoided all DB writes/DDL/DML? | Yes |
| SC7 Avoided patching source / prior KB docs? | Yes (only 3 new reports) |
| SC8 Avoided technical design? | Yes (design-direction items flagged gated) |
| SC9 Kept all blockers open? | Yes |
| SC10 Created only the three allowed reports? | Yes |
| SC11 Re-read all three reports after writing? | Yes |
No self-check failed.
10. Ready for GPT/Codex review
Yes. Recommended chain: GPT reviews R1a + R2a + this execution report → if accepted, Codex adversarial control review → Owner decides R1a-OD-1/2/3, R2a-OD-1/2/3, and OD-8: whether to open write-enabled R1, R2, both, or continue read-only (e.g. authorize the further read-only provenance-source / producer-design studies).
Headline for the Owner: Neither package is a broken runner. R1a = a healthy KG runner held deliberately fail-closed (5 owner-gates) with no provenance source-of-truth to backfill from. R2a = certification was a manual one-shot bootstrap that was never turned into a standing producer/runner. Both "fixes" are net-new build/design under governance — not a restart — and none is authorized by this read-only run. Default disposition: HOLD; PASS ≠ Owner authorization.