KB-16BA

01 — KB Architecture Survey — Queue/Outbox/Event Materials

8 min read Revision 1
surveykbarchitectureoutboxeventp3d4c0xp3d4c1dieu44

01 — KB Architecture Survey

Date: 2026-05-26 | Scope: What the KB already says about queue/outbox/event mechanisms.


§1. Documents read

# Doc Status / age Key contribution
1 dieu44-trien-khai/design/23-p3d4c0x-universal-event-outbox-notification-architecture.md rev1 (2026-05-08) DESIGN — read-only review 5-layer pipeline; 9-domain × 7-stream taxonomy; event envelope; recipient/subscription model; lifecycle states; grouping/debounce; signal channels; candidate Điều 45 note (deferred decision).
2 dieu44-trien-khai/prompts/23-p3d4c1-staging-outbox-worker-notification-implementation-prompt.md rev3 PROMPT — awaiting dispatch Staging + worker + pg_cron birth-trigger implementation prompt; assumes pg_cron will be installed; debounce 90s; stable_key = COALESCE(source_document_ref, import_batch_ref, correlation_id).
3 dieu44-trien-khai/prompts/23-p3d4c1u-universal-core-implementation-prompt.md rev7 PROMPT — awaiting dispatch Universal core implementation prompt + system_issues PoC immediate-only; pg_cron NOT required for PoC; idempotency tightened.
4 dieu44-trien-khai/design/07b-iu0-packaging-crud-vector.md DRAFT — IU-0 module B 9-layer packaging incl. outbox layer; CRUD hot path; vector architecture (vector layer separate from event layer).
5 dieu44-trien-khai/design/09-iu0-minimal-implementation-planning-pack.md rev7 PLANNING Schema sketch non-normative; outbox as part of IU minimal infra.
6 dieu44-trien-khai/v0.6-iu-core-110500x-apply-operator-alias-d30-d31-test/ OPERATOR RUNTIME (live) Operator alias model wrapping creator fns; explicit gates; staging-shape assertions; verify_cut verdict contract.
7 dieu44-trien-khai/v0.6-iu-core-cutting-operator-start-here/ START-HERE Operator-invoked workflow; no automatic scheduler.
8 current-state/reports/pg-qdrant-sync-implementation DEPLOYED 2026-04-05 LISTEN/NOTIFY is in production for PG→Qdrant sync (separate daemon pg_vector_listener.py). Independent of event_outbox.

§2. Architectural commitments already on the books

Drawn from doc #1 (23-P3D4C0X rev1) and doc #4 (07b), which are the closest things to a queue/outbox architecture today:

2.1 Five-layer pipeline (already partially live)

L1 FACT CAPTURE     trigger / fn → 1 INSERT (hot path, O(1))
L2 PROCESSING       worker reads pending, debounces, groups → emits durable
L3 DURABLE EVENT    event_outbox + event_read
L4 ROUTING          event_subscription + view → recipients
L5 EXPOSURE         Directus collection / SQL fn / Nuxt assembly / LISTEN-NOTIFY (Phase 2 eval) / webhook (Phase 3+)

The substrate inventory in 02-… confirms L1–L4 are live to varying degrees; L5 LISTEN/NOTIFY is in use for a different pipeline (PG→Qdrant).

2.2 Taxonomy (already CHECK-enforced live)

  • Domains (9): iu, birth_registry, governance, tac, kg, system, dot, health, piece. Verified in event_outbox_event_domain_check.
  • Streams (7): comment, review, update, birth, task, alert, health. Verified in event_outbox_event_stream_check.
  • Delivery lanes (2): immediate, delayed. Verified in event_outbox_delivery_lane_check.
  • Severities (3): info, warning, critical (nullable).
  • Payload classification (2): safe_metadata, restricted.
  • Payload prohibited keys (10): body, content, raw, vector, embedding, secret, token, password, ssn, personal_data. Verified in event_outbox_safe_payload_check.

2.3 Idempotency model

Document #1 §C proposes UNIQUE (event_domain, event_type, event_subject_table, event_subject_ref) WHERE event_subject_ref IS NOT NULL. Live event_outbox schema follows the spirit — although the exact unique-index list was not re-verified in this survey (see open question Q5 in 08-…).

For consumers: iu_auto_instantiate_event_log.idempotency_key is used as the dedup key (33 successful idempotent invocations on record).

2.4 Grouping / stable_key

source_document_ref, import_batch_ref, correlation_id form a COALESCE chain. The live event_outbox carries correlation_id; per-event source_document_ref / import_batch_ref columns from the design proposal are NOT present in the live event_outbox schema today — they live only in event_pending's correlation_id + capture_payload if needed.

2.5 PG-native exploitation checklist (doc #1 §N)

  • AFTER INSERT triggers ✅
  • Partial index WHERE processed_at IS NULL ✅ (intended for event_pending)
  • pg_cron ❌ NOT INSTALLED — design assumed it would be; rev7 of prompt #3 walked back the requirement for the immediate-lane PoC.
  • pg_try_advisory_lock ⚠️ not seen in any function body grepped (replaced by dot_iu_runtime_lease table-based lease).
  • LISTEN/NOTIFY ⚠️ used in fn_kb_notify_vector_sync for Qdrant only; not used by the universal event substrate.
  • Partial UNIQUE for idempotency ✅
  • Config namespace via dot_config ✅ (9 keys with event.*, iu_core.*, piece_event_runtime.* prefixes).

2.6 Inclusion criteria (doc #1 §M) — already law-quality

The "governance-significant" rubric is fully drafted in doc #1 §M.3 (3-question gate). It has not been ratified as law text. This is one of the strongest candidates to ratify into the queue law because it defines what belongs in the queue.


§3. What KB does NOT yet cover

  • Generic non-event jobs. All current docs are event/outbox-shaped. There is no design pack for a "job_outbox" or "worker_jobs" table that would carry arbitrary worker payloads (Hermes batch, MOT workflow steps, vector reindex, schedule-driven sweeper). The cutting flow uses dot_iu_command_run for ledger but not for queuing.
  • Scheduler under no-pg_cron. The KB has many references to "if pg_cron available" or "pg_cron preflight" but no canonical doc on how external invocation is contracted (who calls fn_iu_route_worker_run, on what cadence, with what authority). This is implicit / operational, not written.
  • Job priority + back-pressure. The event substrate uses delivery_lane ∈ {immediate, delayed}. There is no per-event priority, no per-domain quota, no back-pressure design. Volume to date does not require it but a system-wide queue might.
  • Trigger-IN / Trigger-OUT bridge spec. The mission lists this but the existing docs treat trigger-IN (PG triggers feeding events) and trigger-OUT (events causing actions) inconsistently. iu_sql_event_route is the trigger-IN bridge (dry-run); fn_iu_auto_instantiate_from_event is one trigger-OUT consumer; no unified spec.
  • Cross-DB / cross-host queueing. All current substrate is single-PG-instance. No design for federated queues, sharding, or partitioning by tenant.

§4. What KB explicitly says about a new law

Doc #1 §K (Law Ownership Proposal):

Khuyến nghị: KHÔNG ban hành luật mới ngay. P3D4C0X ghi note "candidate Điều 45 Universal Event/Signal Law" rồi quyết sau Phase 2 PoC.

Phase 2 PoC = the substrate now live. The deferred decision can now be revisited. The KB has thus already pre-flagged the candidate-law slot; this survey simply confirms the precondition is met.


§5. KB drift watch

Two areas where the KB design has drifted from live PG:

  1. Event envelope. Doc #1 §C proposes columns causation_id, source_document_ref, import_batch_ref, source_function, event_subject_type, processed_at that are not present in live event_outbox. Live schema is leaner: event_subject_table, event_subject_ref, correlation_id only.
  2. IU notification compat (doc #1 §I). Live iu_notification_event is empty (0 rows). Either runtime was reset, or compat layer was never wired. Worth re-confirming what was deprecated.

These drifts are noted but not resolved here — they belong in the design pack that follows this survey.

Back to Knowledge Hub knowledge/dev/laws/dieu44-trien-khai/v0.6-system-wide-pg-native-queue-law-readiness-survey/01-kb-architecture-survey.md