00 — Điều 45 Full Queue Orchestration Design Pack DP1-DP7 — Summary
00 — Điều 45 Full Queue Orchestration Design Pack DP1–DP7 — Summary
Date: 2026-05-26 Mode: DESIGN-ONLY (no DDL, no DML, no migration, no production mutation, no extension install, no worker start, no Qdrant touch, no MARK/CUT alias change, no Điều 45 substance change, no PostgreSQL upgrade). Law anchor:
knowledge/dev/laws/dieu45-pg-native-queue-and-task-orchestration-law.md(v1.0 BAN HÀNH 2026-05-26, rev 21). Substrate state at design time: PostgreSQL 16.13;event_outbox133,778 rows;event_read133,439;event_type_registry31; workeriu_outbound_defaultlast_run_at 2026-05-22 11:31 (age ≈ 341,659s ≈ 95h — post-enactment §15.5 violation, see DP4);pg_cronNOT installed; no extension changed during this pack. Scope: All seven sub-design packs (DP1 scheduler, DP2 job substrate, DP3 retry/lease/DLQ, DP4 NOTIFY/heartbeat, DP5 trigger-in/out vocab, DP6 subscription/routing/executor boundary, DP7 partition/retention/archive) plus five cross-cutting design surfaces (MARK/CUT mapping, MOT mapping, customer-care/email/message forward scope, event_outbox transition plan, implementation roadmap).
§0. Hard Gate 0 verification
mode:
design_only: true
DDL_DML_allowed: false
migration_allowed: false
production_mutation_allowed: false
live_pg_read_only_survey_allowed: true
KB_design_write_allowed: true
verified:
no_table_created: true
no_check_constraint_changed: true
no_function_body_modified: true
no_trigger_added: true
no_extension_installed: true
no_dot_config_mutated: true
no_worker_started: true
no_qdrant_call: true
no_mark_cut_alias_touched: true
no_law_text_change: true
no_start_here_patch: true
no_pg_upgrade: true
If a live read-only survey were unavailable, the design would still stand because the KB survey evidence (v0.6-system-wide-pg-native-queue-law-readiness-survey/) is sufficient. The live re-survey at design time is a confirmation, not a dependency.
§1. Top-line design verdict — per DP
| DP | Headline decision | Rationale (one line) |
|---|---|---|
| DP1 — scheduler | Hybrid: external-poll cadence is primary; LISTEN/NOTIFY is opt-in wake-up bridge; pg_cron deferred to its own amendment. |
Honours §5.4 (no pg_cron) while removing the silent-gap class of bug via a documented external cadence + DP4 heartbeat. |
| DP2 — job substrate | New job_queue table layer distinct from event_outbox; same row carries source_ref/target_ref/payload_ref/status/lease/retry; minimum 9-state work state machine per §6.7. |
§6.6 explicitly forbids assuming event_outbox covers job needs; long-running Agent/Hermes/MOT/DOT work needs a distinct execution lifecycle. |
| DP3 — retry/lease/DLQ | FOR UPDATE SKIP LOCKED claim; dot_iu_runtime_lease-style lease per job kind; max_attempts + backoff sourced from dot_config namespace queue.retry.*; stale-lease reaper as a job kind; DLQ widened from event-only to job+event. |
§7, §8 invariants; lease primitive already live. |
| DP4 — NOTIFY/worker/heartbeat | Per-executor heartbeat row in a new queue_heartbeat table; queue_worker_silent event emit per §15.4; NOTIFY bridge optional per channel queue_wake_<domain> (signal-only, never SoT). |
§5.1 / §5.3 forbid NOTIFY-as-SoT; §15.5 makes silent gap a violation. |
| DP5 — trigger-in/out vocab | Widen iu_sql_event_route.target_event_domain CHECK to the §6.1 9-domain set; trigger-in remains SQL→event; trigger-out is consumer_registry (new design table) mapping event_type → executor + idempotency_key_strategy. |
§12 substrate already prototype; just generalised. |
| DP6 — subscription/routing/executor | event_subscription keeps event-side routing; new job_subscription for job-side; executor whitelist ratified per §11.5 (7 executors, MOT NOT executor); recipient resolution = exact match → agency match → role match → broadcast-fallback flagged as warn-only. |
§11.5 + §13.4 codified. |
| DP7 — partition/retention/archive | Hot tier untouched until 5M rows on event_outbox; then RANGE partition by created_at monthly; iu_route_dead_letter keep 365d, iu_staging_record keep 15d (compat), dot_iu_command_run keep 90d (D9 audit baseline); cold archive into archive.event_outbox_yyyymm schema. |
Volume projection 131k→1M in 8 days at system/issue_opened rate. |
§2. Cross-cutting design verdicts
| Surface | Decision |
|---|---|
| MARK/CUT mapping | Each stage of the cutting pipeline (copy_to_staging → mark → verify_mark → cut → verify_cut → cleanup) becomes one job_queue row whose executor is a registered §11.5 executor; status transitions follow §6.7 work state machine; existing operator aliases (fn_iu_op_*) become the executor entrypoints with no signature change. See doc 10. |
| MOT mapping | MOT generates a job graph: one job_queue row per step + a parent job_workflow row; MOT does NOT claim or run rows (it is not an executor per §11.5 / §13.4); workflow lifecycle is derived from constituent job statuses; MOT runtime substrate is design-seam only at this pack. See doc 11. |
| Customer-care / email / message | Five forward seams: customer_message_inbound → classify → draft → approve → send → follow_up/escalation; each becomes a job kind in the same substrate; no concrete schema in this pack (Phase 7 of doc 14). See doc 12. |
event_outbox transition |
event_outbox keeps its current shape; event SoT for facts. Jobs live in job_queue. An event MAY produce a job (via consumer registry) and a job completion MAY emit an event (via standard fn_iu_emit_event call). Single direction at any moment, no duplicate SoT. See doc 13. |
| Implementation roadmap | 7 phases; Phase 1 (minimal job substrate) is the only one that opens new tables; each later phase additive. Phase boundaries map 1:1 to the §18.4 sub-design pack list of Điều 45. See doc 14. |
§3. Quality contract — every DP doc carries
Per the prompt's per-DP requirement, each of docs 03–09 contains:
- Goal (one paragraph).
- Current state (what is live, with row counts where applicable).
- Proposed design (narrative + NON-EXECUTABLE schema sketches where helpful).
- Tables / views / functions named (sketch only, never DDL).
- Lifecycle / status (vocab compatible with §6).
- Indexes / performance notes.
- Security / governance.
- Rollback / disable concept (a
dot_configflag). - Healthcheck / observability per §15.
- Compatibility with Điều 45 v1.0 (specific section references).
- Implementation prerequisites.
- Open questions routed to Council / next pack.
Every NON-EXECUTABLE schema sketch is marked:
NON-EXECUTABLE DESIGN SKETCH — DO NOT APPLY
§4. Compliance with Điều 45 v1.0 invariants
| Invariant (§ in law) | Carried by | Compliance posture |
|---|---|---|
queue_carries_signal_not_data (§4) |
DP2 payload_ref rule; DP5 trigger-in payload_contract jsonb ref-only |
✅ enforced |
event_vs_job_distinction (§6.6) |
DP2 separate job_queue substrate |
✅ enforced |
work_state_machine (§6.7) |
DP2 9-state minimum on job_queue.status |
✅ enforced |
executor_boundary (§11.5) |
DP6 executor whitelist; MOT NOT executor in doc 11 | ✅ enforced |
MOT_is_not_executor (§13.4) |
Doc 11 mapping forbids MOT claiming rows | ✅ enforced |
PG_is_source_of_truth (§5.1) |
All DPs use PG tables; NOTIFY signal-only | ✅ enforced |
LISTEN_NOTIFY_is_wakeup_signal_not_durable_storage (§5.1/§5.3) |
DP4 channel queue_wake_<domain> signal-only |
✅ enforced |
no_vector_for_transient_payload (§14) |
DP2 CHECK forbids vector keys on job_queue.safe_payload | ✅ enforced |
silent_gap_heartbeat_required (§15.5) |
DP4 heartbeat row + alert | ✅ enforced |
retry_dead_letter_required (§8) |
DP3 widens DLQ; reaper as job kind | ✅ enforced |
idempotency_required (§7) |
DP2 unique constraint on job_queue.idempotency_key; DP5 consumer idempotency |
✅ enforced |
status_visibility_required (§15.1) |
Each DP names a view; DP4 adds v_queue_health |
✅ enforced |
§5. Decisions deferred outside this pack
This pack designs; it does not amend Điều 45 nor commit migration. Decisions deliberately NOT made here:
max_attemptsnumeric values per domain (DP3 specifies the contract; Council picks numbers).- pg_cron install date (DP1 carves the gate; DP1-extension pack opens later).
- PG 18 upgrade timing (separate readiness macro).
- Concrete customer/email/message schema (doc 12 is seam-only).
- Concrete MOT runtime schema (doc 11 is seam-only).
- Exact NOTIFY channel naming convention (DP4 proposes; Council ratifies).
§6. Phase-1 substrate footprint (proposed, not applied)
If the roadmap (doc 14) is approved, Phase 1 opens at most:
NEW TABLE job_queue (~12 cols + indexes; signal-only)
NEW TABLE job_workflow (MOT-shaped parent; optional Phase 6)
NEW TABLE queue_heartbeat (1 row per executor; DP4)
NEW TABLE consumer_registry (event_type → executor; DP5)
NEW TABLE job_subscription (DP6 job-side; mirrors event_subscription shape)
NEW VIEW v_queue_health (aggregates all cursors + DLQ + heartbeat)
NEW VIEW v_job_queue_backlog
NEW VIEW v_job_queue_dead_letter_open
NEW dot_config queue.retry.max_attempts.default
NEW dot_config queue.retry.backoff.strategy
NEW dot_config queue.notify.bridge_enabled
NEW dot_config queue.heartbeat.stale_threshold_seconds
No table change to existing event_outbox, event_pending, event_read, event_subscription, event_type_registry, iu_route_*, dot_iu_runtime_lease, iu_core.iu_staging_*, iu_vector_sync_point, dot_iu_command_*.
§7. Top open questions surfaced by this pack
(Full list in doc 15.)
| # | Question | Routed to |
|---|---|---|
| 1 | Approve splitting jobs into job_queue vs piggybacking on event_outbox? |
Council (DP2 decision) |
| 2 | Approve pg_cron deferred indefinitely? |
Council (DP1) |
| 3 | What max_attempts numbers per domain? |
Council (DP3) |
| 4 | Approve queue_wake_<domain> NOTIFY channel naming? |
Council (DP4) |
| 5 | Approve MOT step row = job_queue row (not a new mot_step table)? |
Council (doc 11) |
| 6 | Approve worker silent gap fix priority before/after job substrate? | Council (doc 14 phase order) |
| 7 | Approve broadcast-fallback as warn-only (not deny-by-default)? | Council (DP6) |
| 8 | Partition threshold 5M rows or earlier? | Council (DP7) |
| 9 | iu_notification_event legacy fate — survey Q7 still open. |
Council (doc 13) |
| 10 | Heartbeat write cadence default 30s? 60s? Per-executor override? | Council (DP4) |
§8. Pass criteria status
all_DP1_to_DP7_designed=true
each_DP_has_required_12_sections=true
design_cites_dieu45_v1_0=true
event_vs_job_explicit=true_doc04_doc13
queue_carries_signal_not_data=true_doc04
MOT_is_not_executor=true_doc11
mark_cut_mapping_exists=true_doc10
customer_email_message_seam_exists=true_doc12
event_outbox_transition_plan_exists=true_doc13
implementation_roadmap_exists=true_doc14
no_mutation_verification_exists=true_doc16
kb_list_search_verified=true_doc16
no_production_mutation_occurred=true
verdict=DIEU45_FULL_QUEUE_ORCHESTRATION_DESIGN_PACK_DP1_TO_DP7_PASS
§9. Reading order
- 00-summary.md — this file.
- 01-design-foundation-and-dieu45-compliance.md — anchor to law.
- 02-current-live-substrate-map.md — what is live, with re-survey numbers.
- 03-DP1-scheduler-decision.md through 09-DP7-partition-retention-archive.md — the seven DPs.
- 10-mark-cut-queue-mapping.md, 11-mother-of-task-queue-mapping.md, 12-customer-email-message-forward-scope.md — cross-cutting.
- 13-event-outbox-transition-plan.md — coexistence model.
- 14-implementation-roadmap.md — Phase 1–7.
- 15-risk-register-and-open-questions.md — what's still uncertain.
- 16-no-mutation-verification.md — proof of safety.
- 17-carry-forward.md — handoff to next pack.
Design-only pack. Authored by Claude Opus 4.7 (1M) on 2026-05-26 under Open Goal Prompt Guide v1.2. No production change. Returns DIEU45_FULL_QUEUE_ORCHESTRATION_DESIGN_PACK_DP1_TO_DP7_PASS pending Council review.