Legacy Vector Stabilization OUTLINE v3.1

Trạng thái: OUTLINE v3.1 — KHÔNG execute | Phiên: S190+ (2026-05-02) Dựa trên: VRC Report rev 1 (Codex) + GPT directives (PG CRUD Hot Path Contract + durable outbox queue) v3.1 polish: Q-L5/6/7 chốt thành defaults (worker poll 30s, max retry 3, dead-letter manual+alert). No-execute boundary rõ.

---

§0. PG CRUD Hot Path Contract (GPT chốt — bắt buộc)

NGUYÊN TẮC KIẾN TRÚC: Mọi CRUD path (API, MCP, Directus, PG direct, bulk import) phải nhanh. Hot path = local PG only. Cold path = async worker.

§0.1 15 rules

#	Rule	Diễn giải
HP-1	PG CRUD hot path KHÔNG gọi OpenAI, Qdrant, Langroid, HTTP, FDW, remote service	Zero external dependency trong write response
HP-2	Hot path chỉ: local PG write + validation nhẹ + trigger nhẹ + NOTIFY/outbox INSERT + audit nhẹ	Microseconds, không milliseconds
HP-3	Trigger PHẢI lightweight + idempotent	UPSERT 1 row outbox, không compute nặng
HP-4	Vector projection async, delayed, debounced, retryable, observable	Cold path worker xử lý
HP-5	Nhiều edit trong 120s quiet window → gom thành 1 embedding job	Debounce by document_id (legacy) / unit_version_id (IU)
HP-6	Metadata-only update (title/tags, body không đổi) → KHÔNG enqueue embedding	content_hash gate
HP-7	Worker chỉ embed bản committed mới nhất	Skip intermediate versions
HP-8	Worker idempotent: cùng key process lặp lại = safe	Không duplicate vectors
HP-9	Worker có retry / exponential backoff / dead-letter	OpenAI fail → retry 3x → dead-letter
HP-10	Worker expose queue lag, pending count, failed count, last success, oldest pending age	Observable
HP-11	Search surface vector freshness: current / pending / stale / error / retired	User biết data freshness
HP-12	PG là source of truth ngay sau edit; vector search = eventual consistency (2-3 phút)	Nội dung đã có trong context AI/Agent
HP-13	Direct PG writes (Agent SQL) phải đi qua cùng outbox/trigger path	Không bypass vector sync
HP-14	Bulk edit/import KHÔNG enqueue per-row embedding ngay — batch/debounce	Tránh flood worker
HP-15	Không cleanup/reindex/bulk embedding trước khi có Qdrant snapshot backup	Safety first

§0.2 Hot path vs Cold path

Aspect	Hot Path (CRUD response)	Cold Path (async worker)
Latency	<10ms	1-30s per job (OpenAI API)
External calls	❌ KHÔNG	✅ OpenAI + Qdrant
PG writes	✅ Data + outbox UPSERT	✅ Update vector_status
Blocking CRUD	❌ KHÔNG BAO GIỜ	n/a
Failure impact	Transaction rollback PG	Job retry/dead-letter

§0.3 Evidence align

Precedent	Principle	Align
Đ43 FDW optimize	Hot path local-only, cold path heavy work	✅ HP-1, HP-2
KB Protection Phase 1	Triggers zero-impact write flow	✅ HP-3
TD-131	API CRUD sync vector → drift khi bypass	✅ HP-13 (fix via outbox)
VRC	Listener embed ngay, không delay	✅ HP-4, HP-5 (fix via outbox + worker)

---

§1. Do-not-touch Invariants (kế thừa VRC)

Invariant	Why	Approval if change
Collection production_documents	Active KB search — 11052 points	APR + rollback
Dimension 1536 / Cosine	Match embedding model text-embedding-3-small	APR + rollback
Payload keys document_id, content	Search/audit/delete depend on these	APR + rollback
Chunking 4000/400 overlap	Existing ratios + retrieval quality	APR + rollback
kb_documents table	Agent Data metadata SSOT	APR + rollback
Docker restart unless-stopped	Recovery behavior	APR + rollback
Embedding model text-embedding-3-small	Dimension + quality consistency	APR + rollback
Point ID UUID5 deterministic formula	Audit + dedup depend	APR + rollback
/kb/audit-sync endpoint logic	Orphan/ghost detection depend	APR + rollback
dot-vector-audit script	Scheduled audit depend	APR + rollback
PG function fn_kb_notify_vector_sync	Trigger foundation	APR + rollback
Qdrant storage path /opt/incomex/data/qdrant	Backup + persistence	APR + rollback

---

§2. V5 — Qdrant Backup Schedule (ƯU TIÊN 1)

Evidence

• Script /opt/incomex/scripts/qdrant-backup.sh tồn tại.
• Last snapshot: 2026-04-03 (~1 tháng).
• Crontab: KHÔNG có entry.

Verify read-only

• V5-RO-1: Đọc script backup content.
• V5-RO-2: Kiểm snapshot folder /opt/incomex/backups/qdrant/.
• V5-RO-3: Kiểm crontab tất cả users.

Proposed fix

Thêm crontab: 0 3 * * * /opt/incomex/scripts/qdrant-backup.sh >> /var/log/incomex/qdrant-backup.log 2>&1 Daily 3:00 AM — trước dot-vector-audit 4:30.

Rollback

Xoá crontab entry.

Approval: APR cấp low.

---

§3. V1 — PG Trigger + Async Worker — Durable Outbox Queue

§3.1 Vấn đề kép

• V1-a: Trigger mất → PG direct writes bypass vector sync.
• V1-b: Listener embed ngay → CRUD chậm nếu trigger restored. → Fix V1-b (outbox worker) TRƯỚC V1-a (restore trigger).

§3.2 Verify read-only

• V1-RO-1: Xác nhận function source.
• V1-RO-2: Xác nhận 0 triggers.
• V1-RO-3: Kiểm listener alive.
• V1-RO-4: Kiểm listener code có delay/debounce (kỳ vọng: KHÔNG).

§3.3 DEFAULT: V1-B — Durable Outbox Queue (GPT chốt)

Outbox table (NON-NORMATIVE EXAMPLE)

vector_sync_outbox {
  id: SERIAL PK,
  source_kind: enum('kb_document', 'information_unit'),
  source_id: TEXT NOT NULL,
  operation: enum('upsert', 'delete'),
  content_hash: TEXT,
  first_seen_at: TIMESTAMPTZ DEFAULT now(),
  last_seen_at: TIMESTAMPTZ DEFAULT now(),
  earliest_run_at: TIMESTAMPTZ DEFAULT now() + interval '120 seconds',
  status: enum('pending','processing','done','failed','dead_letter') DEFAULT 'pending',
  attempts: INT DEFAULT 0,
  max_attempts: INT DEFAULT 3,
  locked_at: TIMESTAMPTZ,
  locked_by: TEXT,
  last_error: TEXT,
  completed_at: TIMESTAMPTZ
}

Enqueue logic

Mọi write path: complete PG CRUD → IF content_hash changed → UPSERT outbox (bump earliest_run_at = now()+120s on repeat) → return CRUD response immediately. Metadata-only = no enqueue. Delete = 30s delay.

Worker logic

Poll 30s → SELECT due jobs FOR UPDATE SKIP LOCKED → fetch latest committed content → embed → upsert Qdrant → mark done. Fail → retry exponential backoff (3 max) → dead-letter.

Queue observability

queue_pending_count, queue_failed_count, queue_oldest_pending_age, queue_last_success, queue_lag, dead_letter_count.

§3.4 V1-C — FALLBACK ONLY

V1-C (mark pending + cron) chỉ dùng khi V1-B không khả thi. Warning: AFTER trigger UPDATE cùng table = recursion risk. Safe only via BEFORE trigger SET NEW.vector_status guard. Nhược: no retry/backoff/dead-letter/audit.

§3.5 Thứ tự

1. Create outbox table (DDL)
2. Implement worker
3. Verify worker (manual outbox insert → embed after 120s)
4. Modify API: replace inline embed → outbox UPSERT
5. Verify API path
6. Restore PG trigger (UPSERT outbox, not embed)
7. Verify trigger end-to-end
8. Verify debounce (3 updates 60s → 1 embed)
9. Monitor 24h stable

§3.6 Rollback

Per step reversible. Full: quay về trạng thái hiện tại (API inline embed, no trigger).

§3.7 Approval: APR cấp HIGH.

---

§4. V2 — dot-vector-audit Failing

Evidence

Script gọi localhost:8000 sai (Agent Data trong Docker). Log: FAIL connection refused. --heal = write.

Proposed fix: Option V2-B tách heal khỏi cron (report-only daily, heal manual). APR medium.

---

§5. V3 — 47 Orphan + 7 Ghost

Cleanup CHỈ SAU V5+V1+V2 PASS + Qdrant snapshot. APR medium.

---

§6. V4 — Health Critical

Hệ quả V3. Fix V3 → recheck. Update health endpoint logic to check outbox queue metrics.

---

§7. Thứ tự fix v3.1

V5 (backup) → V1 (outbox table + worker + API modify + trigger) → V2 (audit URL) → V3 (cleanup) → V4 (recheck)

Order	Item	Pre-condition	APR
1	V5 Backup schedule	V5-RO PASS	Low
2	V1 Outbox + worker + API + trigger	V5 PASS + design review	HIGH
3	V2 Audit URL (V2-B tách heal)	V2-RO PASS	Medium
4	V3 Orphan/ghost cleanup	V5+V1+V2 PASS + snapshot	Medium
5	V4 Health recheck + queue metrics	V3 PASS	(auto)

---

§8. Defaults chốt (GPT + User confirmed)

Parameter	Value	Status
Default option	V1-B outbox queue	✅ GPT chốt
Quiet window	120s, configurable 180s	✅ GPT chốt
Delete delay	30-60s async	✅ GPT chốt
Mọi write path	Enqueue-only	✅ GPT chốt
Worker poll interval	30s	✅ GPT chốt
Max retry	3 (exponential backoff)	✅ GPT chốt
Dead-letter handling	Manual review + alert	✅ GPT chốt

---

§9. No-execute Statement

OUTLINE/RUNBOOK only. KHÔNG execute:

• KHÔNG tạo trigger / table / cron.
• KHÔNG patch code / restart container.
• KHÔNG cleanup orphan / reindex ghost.
• KHÔNG đổi Qdrant collection / payload / chunking.
• KHÔNG gọi /kb/audit-sync với auto_heal=true.
• KHÔNG mở P44-6.

Chờ User + GPT review → APR → giao Agent từng step §7.

---

Legacy Vector Stabilization OUTLINE v3.1 | S190+ (2026-05-02) | Dựa VRC Report rev 1 + GPT directives | PG CRUD Hot Path Contract 15 rules + V1-B durable outbox default + V5→V1→V2→V3→V4 thứ tự | KHÔNG execute