01 — KB Architecture Survey — Queue/Outbox/Event Materials
01 — KB Architecture Survey
Date: 2026-05-26 | Scope: What the KB already says about queue/outbox/event mechanisms.
§1. Documents read
| # | Doc | Status / age | Key contribution |
|---|---|---|---|
| 1 | dieu44-trien-khai/design/23-p3d4c0x-universal-event-outbox-notification-architecture.md rev1 (2026-05-08) |
DESIGN — read-only review | 5-layer pipeline; 9-domain × 7-stream taxonomy; event envelope; recipient/subscription model; lifecycle states; grouping/debounce; signal channels; candidate Điều 45 note (deferred decision). |
| 2 | dieu44-trien-khai/prompts/23-p3d4c1-staging-outbox-worker-notification-implementation-prompt.md rev3 |
PROMPT — awaiting dispatch | Staging + worker + pg_cron birth-trigger implementation prompt; assumes pg_cron will be installed; debounce 90s; stable_key = COALESCE(source_document_ref, import_batch_ref, correlation_id). |
| 3 | dieu44-trien-khai/prompts/23-p3d4c1u-universal-core-implementation-prompt.md rev7 |
PROMPT — awaiting dispatch | Universal core implementation prompt + system_issues PoC immediate-only; pg_cron NOT required for PoC; idempotency tightened. |
| 4 | dieu44-trien-khai/design/07b-iu0-packaging-crud-vector.md |
DRAFT — IU-0 module B | 9-layer packaging incl. outbox layer; CRUD hot path; vector architecture (vector layer separate from event layer). |
| 5 | dieu44-trien-khai/design/09-iu0-minimal-implementation-planning-pack.md rev7 |
PLANNING | Schema sketch non-normative; outbox as part of IU minimal infra. |
| 6 | dieu44-trien-khai/v0.6-iu-core-110500x-apply-operator-alias-d30-d31-test/ |
OPERATOR RUNTIME (live) | Operator alias model wrapping creator fns; explicit gates; staging-shape assertions; verify_cut verdict contract. |
| 7 | dieu44-trien-khai/v0.6-iu-core-cutting-operator-start-here/ |
START-HERE | Operator-invoked workflow; no automatic scheduler. |
| 8 | current-state/reports/pg-qdrant-sync-implementation |
DEPLOYED 2026-04-05 | LISTEN/NOTIFY is in production for PG→Qdrant sync (separate daemon pg_vector_listener.py). Independent of event_outbox. |
§2. Architectural commitments already on the books
Drawn from doc #1 (23-P3D4C0X rev1) and doc #4 (07b), which are the closest things to a queue/outbox architecture today:
2.1 Five-layer pipeline (already partially live)
L1 FACT CAPTURE trigger / fn → 1 INSERT (hot path, O(1))
L2 PROCESSING worker reads pending, debounces, groups → emits durable
L3 DURABLE EVENT event_outbox + event_read
L4 ROUTING event_subscription + view → recipients
L5 EXPOSURE Directus collection / SQL fn / Nuxt assembly / LISTEN-NOTIFY (Phase 2 eval) / webhook (Phase 3+)
The substrate inventory in 02-… confirms L1–L4 are live to varying degrees; L5 LISTEN/NOTIFY is in use for a different pipeline (PG→Qdrant).
2.2 Taxonomy (already CHECK-enforced live)
- Domains (9):
iu, birth_registry, governance, tac, kg, system, dot, health, piece. Verified inevent_outbox_event_domain_check. - Streams (7):
comment, review, update, birth, task, alert, health. Verified inevent_outbox_event_stream_check. - Delivery lanes (2):
immediate, delayed. Verified inevent_outbox_delivery_lane_check. - Severities (3):
info, warning, critical(nullable). - Payload classification (2):
safe_metadata, restricted. - Payload prohibited keys (10): body, content, raw, vector, embedding, secret, token, password, ssn, personal_data. Verified in
event_outbox_safe_payload_check.
2.3 Idempotency model
Document #1 §C proposes UNIQUE (event_domain, event_type, event_subject_table, event_subject_ref) WHERE event_subject_ref IS NOT NULL. Live event_outbox schema follows the spirit — although the exact unique-index list was not re-verified in this survey (see open question Q5 in 08-…).
For consumers: iu_auto_instantiate_event_log.idempotency_key is used as the dedup key (33 successful idempotent invocations on record).
2.4 Grouping / stable_key
source_document_ref, import_batch_ref, correlation_id form a COALESCE chain. The live event_outbox carries correlation_id; per-event source_document_ref / import_batch_ref columns from the design proposal are NOT present in the live event_outbox schema today — they live only in event_pending's correlation_id + capture_payload if needed.
2.5 PG-native exploitation checklist (doc #1 §N)
- AFTER INSERT triggers ✅
- Partial index
WHERE processed_at IS NULL✅ (intended for event_pending) - pg_cron ❌ NOT INSTALLED — design assumed it would be; rev7 of prompt #3 walked back the requirement for the immediate-lane PoC.
- pg_try_advisory_lock ⚠️ not seen in any function body grepped (replaced by
dot_iu_runtime_leasetable-based lease). - LISTEN/NOTIFY ⚠️ used in
fn_kb_notify_vector_syncfor Qdrant only; not used by the universal event substrate. - Partial UNIQUE for idempotency ✅
- Config namespace via
dot_config✅ (9 keys withevent.*,iu_core.*,piece_event_runtime.*prefixes).
2.6 Inclusion criteria (doc #1 §M) — already law-quality
The "governance-significant" rubric is fully drafted in doc #1 §M.3 (3-question gate). It has not been ratified as law text. This is one of the strongest candidates to ratify into the queue law because it defines what belongs in the queue.
§3. What KB does NOT yet cover
- Generic non-event jobs. All current docs are event/outbox-shaped. There is no design pack for a "job_outbox" or "worker_jobs" table that would carry arbitrary worker payloads (Hermes batch, MOT workflow steps, vector reindex, schedule-driven sweeper). The cutting flow uses
dot_iu_command_runfor ledger but not for queuing. - Scheduler under no-pg_cron. The KB has many references to "if pg_cron available" or "pg_cron preflight" but no canonical doc on how external invocation is contracted (who calls
fn_iu_route_worker_run, on what cadence, with what authority). This is implicit / operational, not written. - Job priority + back-pressure. The event substrate uses
delivery_lane ∈ {immediate, delayed}. There is no per-event priority, no per-domain quota, no back-pressure design. Volume to date does not require it but a system-wide queue might. - Trigger-IN / Trigger-OUT bridge spec. The mission lists this but the existing docs treat trigger-IN (PG triggers feeding events) and trigger-OUT (events causing actions) inconsistently.
iu_sql_event_routeis the trigger-IN bridge (dry-run);fn_iu_auto_instantiate_from_eventis one trigger-OUT consumer; no unified spec. - Cross-DB / cross-host queueing. All current substrate is single-PG-instance. No design for federated queues, sharding, or partitioning by tenant.
§4. What KB explicitly says about a new law
Doc #1 §K (
Law Ownership Proposal):Khuyến nghị: KHÔNG ban hành luật mới ngay. P3D4C0X ghi note "candidate Điều 45 Universal Event/Signal Law" rồi quyết sau Phase 2 PoC.
Phase 2 PoC = the substrate now live. The deferred decision can now be revisited. The KB has thus already pre-flagged the candidate-law slot; this survey simply confirms the precondition is met.
§5. KB drift watch
Two areas where the KB design has drifted from live PG:
- Event envelope. Doc #1 §C proposes columns
causation_id,source_document_ref,import_batch_ref,source_function,event_subject_type,processed_atthat are not present in liveevent_outbox. Live schema is leaner:event_subject_table,event_subject_ref,correlation_idonly. - IU notification compat (doc #1 §I). Live
iu_notification_eventis empty (0 rows). Either runtime was reset, or compat layer was never wired. Worth re-confirming what was deprecated.
These drifts are noted but not resolved here — they belong in the design pack that follows this survey.