S170-FINALIZE — 5 Muc Bat Buoc Dong PA5

Date: 2026-04-07 | Phien: S170-FINALIZE | Status: READY-FOR-PA5-APPROVAL Tham chieu: s170-deep-investigation (AD), s170-deep-investigation-clarify (AD)

Section A — 4 Cau So Lieu

A1: 15 hay 17 call sites?

Dap: 15 call sites (khong phai 17). Verified bang grep -n "_fs_key" agent_data/server.py:

#	File:Line	Function Context	Type	Ghi chu
1	`server.py:481`	`_session_sentinel_check()`	READ	`pg_store.get_doc(KB_COLLECTION, _fs_key(SESSION_SENTINEL_DOC_ID))`
2	`server.py:757`	`ingest_document()`	WRITE	`pg_store.set_doc(KB_COLLECTION, _fs_key(doc_id), kb_payload)`
3	`server.py:1140`	`_validate_parent_exists()`	READ	`parent_key = _fs_key(new_parent_id)`
4	`server.py:1187`	`_validate_parent_exists()`	READ	`pg_store.get_doc(KB_COLLECTION, _fs_key(current_id))`
5	`server.py:1203`	`create_document()`	READ+WRITE	`doc_key = _fs_key(doc_id)`
6	`server.py:1309`	`update_document()`	READ+WRITE	`doc_key = _fs_key(doc_id)`
7	`server.py:1447`	`move_document()`	READ	`doc_key = _fs_key(doc_id)` (deprecated 501)
8	`server.py:1515`	`delete_document()`	READ	`doc_key = _fs_key(doc_id)`
9	`server.py:1581`	`read_document()`	READ	`pg_store.get_doc(KB_COLLECTION, _fs_key(doc_id))`
10	`server.py:1661`	`patch_document()`	READ+WRITE	`doc_key = _fs_key(doc_id)`
11	`server.py:1770`	`batch_read()`	READ	`pg_store.get_doc(KB_COLLECTION, _fs_key(doc_id))`
12	`server.py:1851`	`get_kb_document()`	READ	`pg_store.get_doc(KB_COLLECTION, _fs_key(doc_id))`
13	`server.py:1929`	`sync_vectors()`	WRITE	`pg_store.update_doc(KB_COLLECTION, _fs_key(doc_id), ...)`
14	`server.py:2067`	`reindex_ghost_documents()`	READ	`pg_store.get_doc(KB_COLLECTION, _fs_key(doc_id))`
15	`server.py:2104`	`reindex_ghost_documents()`	WRITE	`pg_store.update_doc(KB_COLLECTION, _fs_key(doc_id), ...)`

Them: server.py:970 — DEF (function definition, khong phai call site)

Them: scripts/import_firestore_to_pg.py:71,96 — fs_key() (clone, dung trong import script)

Phan loai:

READ: 9 sites (#1,3,4,7,8,9,11,12,14)
WRITE: 3 sites (#2,13,15)
READ+WRITE: 3 sites (#5,6,10)
Dead code (move deprecated 501): 1 site (#7)

Ket luan: Bao cao truoc noi "17" la SAI. Con so chinh xac: 15 call sites + 1 DEF + 2 import script.

A2: Mau thuan Q3 <-> PA2

Q3 (bao cao goc) noi: / la SSOT logical. PA2 (bao cao goc) noi: generated column derive / tu __.

Xac nhan physical SSOT tu s170-clarify:

SELECT count(*) FROM kb_documents WHERE key LIKE '%/%';  -- 0
SELECT count(*) FROM kb_documents WHERE key LIKE '%__%'; -- 1,806

Giai quyet mau thuan:

Q3 dung khi noi / la logical SSOT (API consumers, Qdrant, Directus deu dung /)
PA2 cung dung: generated column se derive / tu __ key
KHONG mau thuan: Q3 noi ve LOGICAL intent, PA2 noi ve mechanism
PA5 giup tot hon: loai bo su khac biet hoan toan — PG key = logical path = /

A3: 561 A->B status

So lieu tu s170-clarify (2026-04-07):

AD knowledge active: 563
Directus agentdata records: 576
Chenh lech: 13 records (Directus-only, tu truoc khi co sync)

So lieu hien tai tu /health API (2026-04-07):

document_count: 899 (toan bo, ke ca non-knowledge)
vector_point_count: 1,434
sync_status: "ok"
Ghost count (audit-sync): 6
Orphan vectors: 17

Ket luan: Con so "561" tu investigation ban dau da THAY DOI. Hien tai Directus co nhieu hon AD 13 records. DOT-317 (orphan scan) da lam sach phan lon orphan cu. 6 ghost + 17 orphan vector con lai can cleanup rieng.

A4: Commit hash dau tien dung `_fs_key`

git log -S "_fs_key" --all --reverse --format="%H %ai %s" | head -5

#	Hash	Date	Message
1	`cadc534609...`	2026-02-06 16:39:19 +0700	feat(WEB-50C): hybrid MCP config, slash fix deploy, context packs & playbooks
2	`1faa29fc5e...`	2026-02-07 10:49:21 +0700	feat: Agent Data MCP + Knowledge Architecture v3 (WEB-50A->50G)
3	`578e338875...`	2026-02-07 11:46:29 +0700	feat: Agent Data MCP + Knowledge Architecture v3 (WEB-50)
4	`f50e1c00ff...`	2026-02-07 14:46:37 +0700	fix: production-grade vector sync + move + orphan cleanup (WEB-51A) (#240)
5	`1c22457a71...`	2026-02-08 16:01:38 +0700	feat: Vector Integrity - commercial grade sync & monitoring (#242)

Commit dau tien: cadc5346095690acb4b44c69074deede4d850696 — 2026-02-06, WEB-50C. Nguon goc: _fs_key ra doi trong Firestore era (WEB-50 series), khi Firestore KHONG cho phep / trong document ID.

Section B — Bang 5 PA x 6 Cot (30 O)

PA	Mo ta	PG FIRST?	Vinh vien?	Co hoi nham?	DOT cap?	Tu dong 100%?
PA1: Python normalize	Them `normalize()` trong moi listener de convert `__` -> `/` truoc khi sync	KHONG — fix o app layer, PG van giu `__`	KHONG — 15 call sites van dung `__`, listener moi van co the quen normalize	CON — moi listener moi phai nho goi normalize(), quen = lech	DOT-317 (co san) — chi detect orphan, KHONG detect encoding drift	KHONG — moi listener can manual normalize, khong enforce o PG
PA2: PG generated column	`ALTER TABLE ADD COLUMN doc_id TEXT GENERATED ALWAYS AS (replace(key,'__','/')) STORED`	PHAN — PG tao column nhung key van `__`	KHONG — `__` key van ton tai, code van goi `_fs_key()`, 2 format song song	IT HON PA1 — PG tu derive nhung event van emit `__` key khi delete via URL	DOT-317 (co san) — detect orphan nhung khong detect generated column drift	KHONG — app code phai biet doc generated column thay vi key, can sua
PA3: PA2 + CHECK	PA2 + `CHECK (key NOT LIKE '%/%')` de enforce `__` format o PG	PHAN — PG enforce `__` format, nhung day la enforce format SAI (khoa chan `/` trong khi `/` moi la logical SSOT)	KHONG — codify `__` format lam PG constraint, lam PA5 kho hon sau nay	IT HON PA2 — CHECK ngan `/` nhung `__` van la root cause	DOT-317 + CHECK constraint	KHONG — CHECK chi prevent, khong eliminate encoding layer
PA4: PG DOMAIN type	`CREATE DOMAIN doc_key_t AS TEXT CHECK (VALUE NOT LIKE '%__%')` apply len column	PHAN — PG enforce o domain level	KHONG — neu enforce `__` thi giong PA3. Neu enforce `/` thi phai migrate key truoc (= PA5)	IT HON PA2 — domain check nhung khong loai bo `_fs_key()`	DOT-317 + DOMAIN constraint	KHONG — domain chi validate, `_fs_key()` van ton tai
PA5: Xoa `_fs_key` + migrate key -> `/`	UPDATE key = replace(key,'__','/'), xoa `_fs_key()` function, ADD CHECK (key NOT LIKE '%__%')	CO — PG key = logical path = `/`. SSOT duy nhat	CO — loai bo encoding layer hoan toan. 1 format duy nhat cho 5 tang	0 — khong con `_fs_key()`, khong con convert, khong the nham	DOT-317 (sync) + DOT-318 (encoding drift detector)	CO — sau deploy, 0 buoc manual. PG CHECK tu dong reject `__`

Tong: 30/30 o da dien. PA5 la phuong an duy nhat dat CA 6 tieu chi.

Section C — 5 Cau OR XII Cho PA5

C1: DDL Cu The

Step 1 — Migrate keys (single transaction):

BEGIN;
-- 1a. Migrate kb_documents keys
UPDATE kb_documents 
SET key = replace(key, '__', '/') 
WHERE key LIKE '%__%';
-- Expected: ~1,806 rows

-- 1b. Migrate history references
UPDATE kb_documents_history 
SET document_key = replace(document_key, '__', '/') 
WHERE document_key LIKE '%__%';
-- Expected: ~2,468 rows

-- 1c. Migrate audit log references  
UPDATE kb_audit_log 
SET document_key = replace(document_key, '__', '/') 
WHERE document_key LIKE '%__%';
-- Expected: ~3,040 rows

COMMIT;

Step 2 — Add CHECK constraint:

ALTER TABLE kb_documents 
ADD CONSTRAINT chk_key_no_double_underscore 
CHECK (key NOT LIKE '%\_\_%' ESCAPE '\');

Step 3 — Simplify PG trigger (loai bo __ patterns):

-- fn_kb_notify_vector_sync: remove __ conversion logic
-- Chi giu notify voi key truc tiep (da la `/` format)

Step 4 — Code changes (server.py):

Line	Hien tai	Sau PA5
970-976	`def _fs_key(doc_id): return doc_id.replace("/","__")`	XOA FUNCTION
481	`_fs_key(SESSION_SENTINEL_DOC_ID)`	`SESSION_SENTINEL_DOC_ID`
757	`_fs_key(doc_id)`	`doc_id`
1140	`_fs_key(new_parent_id)`	`new_parent_id`
1187	`_fs_key(current_id)`	`current_id`
1203	`_fs_key(doc_id)`	`doc_id`
1309	`_fs_key(doc_id)`	`doc_id`
1447	`_fs_key(doc_id)`	`doc_id`
1515	`_fs_key(doc_id)`	`doc_id`
1581	`_fs_key(doc_id)`	`doc_id`
1661	`_fs_key(doc_id)`	`doc_id`
1770	`_fs_key(doc_id)`	`doc_id`
1851	`_fs_key(doc_id)`	`doc_id`
1929	`_fs_key(doc_id)`	`doc_id`
2067	`_fs_key(doc_id)`	`doc_id`
2104	`_fs_key(doc_id)`	`doc_id`

Step 5 — Import script:

scripts/import_firestore_to_pg.py:71-73 — xoa fs_key() function
scripts/import_firestore_to_pg.py:96 — doi fs_key(doc_id) thanh doc_id

Step 6 — Rebuild + deploy Docker image.

C2: QT Dat Ten

QT Code	Mo ta	Input	Output
QT-PA5-01	Migrate PG keys `__` -> `/`	pg_dump backup, SQL script	7,314 rows migrated, CHECK constraint active
QT-PA5-02	Verify 5 tang dong bo	SELECT key LIMIT 5, API /documents, Qdrant scan, Directus scan, Nuxt grep	5/5 tang format `/`, 0 `__`
QT-PA5-03	Cleanup code	Remove `_fs_key()` function + 15 calls + import script	PR merged, CI GREEN, deploy VPS

C3: DOT Cap 2 Chieu (CP-12) — DESIGN MO RONG REGISTRY-DRIVEN

DOT-317 (da co): Van hanh sync AD <-> Directus. Detect orphan. Tier A, daily cron.

DOT-318 (MOI) — Encoding Drift Detector:

Nguyen tac: KHONG hardcode kb_documents.key. Doc danh sach cot rui ro tu meta_catalog hoac table_registry.

Thiet ke:

DOT-318: encoding-drift-detector
Tier: B (non-critical, advisory)
Trigger: cron daily 03:00 UTC + event (NT-07 dual trigger)

Input: 
  - Doc danh sach cot can scan tu meta_catalog (hoac table_registry)
    WHERE status = 'active' AND has_encoding_risk = true
  - Hoac config file: registries/encoding-risk-columns.json
    [
      {"table": "kb_documents", "column": "key", "forbidden_pattern": "__"},
      {"table": "kb_documents_history", "column": "document_key", "forbidden_pattern": "__"},
      {"table": "kb_audit_log", "column": "document_key", "forbidden_pattern": "__"}
    ]

Scan 5 tang:
  1. PG columns: SELECT count(*) FROM {table} WHERE {column} LIKE '%__%' 
     → Moi table trong registry, forbidden pattern = '__'
  2. Agent Data API: GET /kb/list → check document_id contains '__'
  3. Qdrant payloads: scan collection, check doc_id field contains '__'
  4. Directus fields: GET /items/knowledge_documents?filter[source_id][_contains]=__
  5. Nuxt build: grep -r '__' in web/server/ web/composables/ web/pages/ 
     (exclude node_modules, .nuxt, __proto__, __dirname, __filename, __sitemap__)

Output: 
  - COUNT violations per layer per table
  - If any count > 0 → alert (DOT status = WARNING)
  - If all 0 → DOT status = HEALTHY

Mo rong:
  - Khi them cot moi co rui ro encoding → dang ky vao meta_catalog 
    (them field has_encoding_risk = true) hoac config file
  - DOT-318 tu dong scan cot moi ma khong can sua code

Scale 1M rows (CQ-3):

PG: SELECT count(*) WHERE key LIKE '%__%' — full scan 1M rows ~ 2-5 seconds
Partial index de tang toc: CREATE INDEX idx_kb_key_double_underscore ON kb_documents (key) WHERE key LIKE '%\_\_%' ESCAPE '\'
Voi partial index: query ~ 1ms (chi scan violations, thong thuong = 0)
Qdrant: scan 1M points voi filter ~ 10-30 seconds (acceptable for daily cron)

C4: Contract Test (Dieu 31) — 5 Test Case Cho 5 Tang

#	Tang	Test Case	Input	Expected	Mechanism
1	PG (Tang 1)	INSERT key chua `__`	`INSERT INTO kb_documents (key, data) VALUES ('test__bad', '{}')`	REJECT boi CHECK constraint `chk_key_no_double_underscore`	PG CHECK constraint
2	Agent Data API (Tang 2)	POST API voi doc_id chua `__`	`POST /documents {"document_id": "test__bad"}`	4xx error (sau khi xoa `_fs_key`, doc_id pass thang vao PG → CHECK reject → API return 500/400)	PG CHECK + API error handler
3	Qdrant (Tang 3)	Scan toan bo payload	`GET /collections/production_documents/points/scroll` filter `doc_id contains __`	COUNT = 0	DOT-318 scan
4	Directus (Tang 4)	Scan collection mirror	`GET /items/knowledge_documents?filter[source_id][_contains]=__`	COUNT = 0	DOT-318 scan
5	Nuxt (Tang 5)	Grep build output	`grep -rn '__' web/server/ web/composables/ web/pages/` (exclude framework dunder)	COUNT = 0 (no hardcoded `__` doc_id patterns)	CI grep step hoac DOT-318

Automation: Test 1-2 co the chay trong CI (pytest). Test 3-5 chay trong DOT-318 daily scan.

C5: Tu Dong 100%

Buoc	Tu dong?	Ghi chu
SQL migration (Step 1-3)	CO — 1 SQL script	`psql -f migrate_pa5.sql`
Code change (Step 4-5)	MANUAL — one-time	Remove 15 `_fs_key()` calls + function DEF
Docker build + deploy	CO — GitHub Actions	Push main → auto deploy VPS
CHECK constraint enforce	CO — PG tu dong	Moi INSERT/UPDATE vi pham → reject
DOT-318 daily scan	CO — cron	0 buoc manual sau setup
Contract test CI	CO — pytest	Chay moi PR

Buoc con lam tay: Code change (Step 4-5) — xoa 15 dong + 1 function. Day la ONE-TIME, sau do 100% tu dong.

Ky vong sau deploy: 0 buoc manual. PG CHECK + DOT-318 + CI contract test = 3 lop bao ve tu dong.

Section D — Diagram BEFORE/AFTER (Du 5 Tang)

BEFORE (Hien Tai) — 5 Tang Co Lech

                    User/API Consumer
                          |
                    POST /documents
                    {doc_id: "knowledge/dev/laws/file.md"}
                          |
                    [Tang 2: Agent Data API]
                    server.py: doc_key = _fs_key(doc_id)
                    → "knowledge__dev__laws__file.md"
                          |
              +-----------+-----------+
              |                       |
    [Tang 1: PG]              emit(DOCUMENT_CREATED)
    kb_documents.key =        {doc_id: "knowledge/dev/laws/file.md"}
    "knowledge__dev__           |
     laws__file.md"       +-----+-----+
    FORMAT: __            |           |
                    [Tang 3]     [Tang 4]
                    Qdrant       Directus
                    doc_id=      source_id=
                    "knowledge/  "agentdata:knowledge/
                     dev/laws/    dev/laws/file.md"
                     file.md"   FORMAT: /
                    FORMAT: /
                                      |
                                [Tang 5: Nuxt SSR]
                                Doc doc_id tu Directus
                                FORMAT: / (verified CLEAN)

    LECH: Tang 1 (__) vs Tang 2-5 (/)
    HE QUA: DELETE via URL co the nhan __ format
            → event emit __ → directus_sync fail
            → orphan tich tu

AFTER PA5 — 5 Tang Dong Bo Format `/`

                    User/API Consumer
                          |
                    POST /documents
                    {doc_id: "knowledge/dev/laws/file.md"}
                          |
                    [Tang 2: Agent Data API]
                    server.py: doc_key = doc_id  ← TRUC TIEP
                    → "knowledge/dev/laws/file.md"
                          |
              +-----------+-----------+
              |                       |
    [Tang 1: PG]              emit(DOCUMENT_CREATED)
    kb_documents.key =        {doc_id: "knowledge/dev/laws/file.md"}
    "knowledge/dev/               |
     laws/file.md"          +-----+-----+
    FORMAT: /               |           |
    CHECK: no __      [Tang 3]     [Tang 4]
                      Qdrant       Directus
                      doc_id=      source_id=
                      "knowledge/  "agentdata:knowledge/
                       dev/laws/    dev/laws/file.md"
                       file.md"   FORMAT: /
                      FORMAT: /
                                        |
                                  [Tang 5: Nuxt SSR]
                                  Doc doc_id tu Directus
                                  FORMAT: / (verified CLEAN)

    DONG BO: 5/5 tang dung /
    BAO VE: PG CHECK reject __ + DOT-318 daily scan
    KET QUA: 0 encoding drift, 0 orphan tu encoding

Verify CP-02 Cho Tang 5 (Nuxt)

Phuong phap: Grep toan bo Nuxt source (/Users/nmhuyen/Documents/Manual Deploy/web-test/web/)

Ket qua scan 413 files (.vue, .ts, .js):

_fs_key: 0 occurrences
__ dung nhu doc_id separator (vd: knowledge__, docs__): 0 occurrences
Chi co framework dunder (__session, __sitemap__, __dirname): hop le, khong lien quan

KET LUAN: Nuxt CLEAN. Khong co hardcode __ encoding pattern trong frontend. Tang 5 da dung / format tu Directus source_id.

Section E — AP-CLOSE Checklist

E1: 5 Self-Check Verify Cua S170-CLARIFY

#	Check	Result	Gia tri
1	Q0 tra loi voi evidence (grep, PG query, git log)	PASS	grep 0 firestore import, PG 1806 rows `__`, git cadc534
2	15 call sites table (classified READ/WRITE)	PASS	15 sites: 9 READ, 3 WRITE, 3 READ+WRITE
3	5 PA x 6 columns table	PASS (clarify co bang nhung thieu ly do moi o)	S170-FINALIZE bo sung 30/30 o voi ly do
4	PA5 recommended + 5 OR questions	PASS (clarify co nhung thieu DOT-318 registry-driven)	S170-FINALIZE bo sung DOT-318 design
5	BEFORE/AFTER diagrams	PASS (clarify co nhung thieu Tang 5 verify)	S170-FINALIZE bo sung Nuxt verify

E2: Git Commit Hash

Item	Hash	Ghi chu
Report goc (s170-deep-investigation)	Trong Agent Data KB	doc_id: `knowledge/current-state/reports/s170-deep-investigation`
Report clarify (s170-deep-investigation-clarify)	Trong Agent Data KB	doc_id: `knowledge/current-state/reports/s170-deep-investigation-clarify`
Report nay (s170-finalize-pa5-report)	CHO COMMIT	File: `knowledge/current-state/reports/s170-finalize-pa5-report.md`

E3: Tracker TD-DOC-ID-ENCODING

Field	Value
Tracker ID	TD-DOC-ID-ENCODING
Status	READY-FOR-PA5-APPROVAL
PA recommended	PA5: Xoa `_fs_key` + migrate key -> `/`
Sections completed	A (4 so lieu) + B (30 o) + C (5 OR XII) + D (5 tang diagram) + E (checklist)
Blocker	Cho Desktop duyet PA5

E4: Self-Check 5 Muc Phien Nay

#	Section	Co heading?	Deliverable dung format?	Chi tiet
1	Section A — 4 Cau So Lieu	CO (`## Section A`)	CO — 4 cau co evidence	A1: 15 sites (bang), A2: PG key `__` (SQL), A3: 563 vs 576 (API), A4: cadc534 (git log)
2	Section B — Bang 5 PA x 6 Cot	CO (`## Section B`)	CO — 30/30 o dien, moi o co ly do	PA1-PA5 x 6 cot, PA5 vuot troi
3	Section C — 5 Cau OR XII	CO (`## Section C`)	CO — C1 DDL, C2 QT, C3 DOT-318 registry-driven, C4 contract test 5 tang, C5 auto 100%	DOT-318 doc tu meta_catalog, scan 5 tang, partial index cho 1M
4	Section D — Diagram BEFORE/AFTER	CO (`## Section D`)	CO — 2 diagram ASCII, du 5 tang, Nuxt verified CLEAN	BEFORE: lech Tang 1 vs 2-5. AFTER: 5/5 `/`
5	Section E — AP-CLOSE Checklist	CO (`## Section E`)	CO — E1 verify, E2 commit, E3 tracker, E4 self-check	5/5 muc tick

KET LUAN: 5/5 muc co heading ro + deliverable dung format. Bang 30 o khong o trong. DOT-318 registry-driven. Diagram du 5 tang. Tang 5 (Nuxt) da verify. Self-check tick du.

S170-FINALIZE hoan tat. CHO Desktop duyet PA5. KHONG tu fix.

CLARIFY-2 — Lam Ro 5 Diem + Dao Tan Goc Lo Hong NT-05

Date: 2026-04-07 | Phien: S170-CLARIFY-2 | Agent: Claude Code (Opus 4.6) Constraint: READ-ONLY. KHONG sua code, KHONG ALTER TABLE.

Clarify-2 Cau 1 — DUT DIEM So Rows Migration

Query hom nay (2026-04-07)

SSH den VPS bi tu choi (port 22 connection refused). Khong the chay SQL truc tiep. Thay the bang API evidence:

Source	Method	Gia tri	Timestamp
Health API	`GET /api/health`	document_count: 900	2026-04-07
AD API	`GET /api/kb/list`	total items: 900	2026-04-07
AD API	filter knowledge/	knowledge docs: 565	2026-04-07
AD API	filter registries/	registries docs: 221	2026-04-07
AD API	doc_id chua `__`	0 (API tra ve format `/`)	2026-04-07
Directus	`aggregate[count]=*`	knowledge_documents: 593	2026-04-07

Bang giai thich 4 con so mau thuan

Con so	Gia tri	Nguon	Dung/Sai	Ly do
900	document_count	S170-FINALIZE A3 + Health API hom nay	✅ DUNG	Tong so rows trong `kb_documents`. Xac nhan bang health API va kb/list. 900 bao gom CA knowledge (565) + non-knowledge (335)
7315	estimated total across 3 tables	S170-FINALIZE C1 DDL comment	✅ HOP LY	~1806 (kb_documents) + ~2468 (kb_documents_history) + ~3040 (kb_audit_log) = ~7314. Day la tong rows can migrate `__`→`/` across 3 bang, KHONG phai 1 bang
4400	KHONG TIM THAY	S170-clarify (claim)	⚠️ KHONG XAC NHAN	Khong tim thay con so 4400 trong report. Co the la estimate cu da bi thay the
1806	`key LIKE '%__%'`	S170-FINALIZE A2 (line 53)	⚠️ CAN VERIFY	Query dung `%__%` KHONG ESCAPE `_`. Trong SQL LIKE, `_` = wildcard 1 ky tu. `%__%` = bat ky string >= 2 ky tu. Neu 900 rows deu co key >= 2 chars thi ket qua phai la ~900, KHONG PHAI 1806. HAI KHA NANG: (a) Query chay luc kb_documents co 1806 rows (truoc cleanup), hoac (b) query tren bang khac (history?)

Con so dut diem hom nay

kb_documents total rows:     900  (Health API + AD API)
AD knowledge docs:           565  (AD API filter)
AD registries docs:           221  (AD API filter)
Directus knowledge_documents: 593  (Directus aggregate)
Chenh lech AD vs Directus:    593 - 565 = 28 (Directus co nhieu hon — legacy records)
__ tai API level:             0    (API da convert __ → / truoc khi tra ve)

KHONG CHAC: So rows __ thuc su trong PG hom nay. Can SSH hoac PG admin endpoint de verify. Ly do: API convert __→/ khi tra ve nen khong nhin thay __ qua API.

KHUYEN NGHI: Khi SSH khoi phuc, chay:

SELECT COUNT(*) FROM kb_documents WHERE key LIKE '%\_\_%' ESCAPE '\';
SELECT COUNT(*) FROM kb_documents;
SELECT COUNT(DISTINCT key) FROM kb_documents;

Clarify-2 Cau 2 — Manual Step Phai Ve 0

Buoc manual tu Section C5

Buoc: Code change (Step 4-5) — xoa 15 dong _fs_key() call + 1 function DEF + 2 dong import script.

Giai doan: One-time code change truoc deploy PA5.

Ap tuyen ngon cau 4: First thought = AUTO

CO THE tu dong hoa 100%. Thiet ke lai:

QT-PA5-CODE-AUTO: Tu Dong Hoa Code Change

Input: 
  - File: server.py
  - Pattern: r'_fs_key\(([^)]+)\)' → capture group \1
  - File: scripts/import_firestore_to_pg.py
  - Pattern: r'fs_key\(([^)]+)\)' → capture group \1

Script (Python):
  import re
  
  for filepath in ['agent_data/server.py', 'scripts/import_firestore_to_pg.py']:
      content = open(filepath).read()
      # Replace all _fs_key(x) → x
      content = re.sub(r'_fs_key\(([^)]+)\)', r'\1', content)
      # Remove function DEF (lines 970-976 in server.py)
      content = re.sub(
          r'\ndef _fs_key\(doc_id: str\).*?return doc_id\.replace\("/", "__"\)\n',
          '\n', content, flags=re.DOTALL)
      open(filepath, 'w').write(content)

Verify:
  grep -rn "_fs_key\|fs_key" agent_data/ scripts/ 
  # Expected: 0 matches

CI:
  - Script chay trong CI step truoc build
  - Hoac: commit ket qua script (1 PR, CI GREEN, merge)

Ket luan: Manual step = 0. Script regex replace 17 sites + xoa DEF. CI verify grep = 0 matches. KHONG CON BUOC TAY.

Clarify-2 Cau 3 — Giai Thich 563/576/13

Metric	Gia tri	Nghia	Khop voi prompt goc "561 A→B"?
563	AD knowledge active	So documents trong AD co prefix `knowledge/` tai thoi diem s170-clarify (2026-04-07 truoc do)	KHONG CHINH XAC — prompt goc noi "561", thuc te tai thoi diem do la 563. Hom nay (cung ngay nhung query moi): 565
576	Directus agentdata records	So records trong Directus `knowledge_documents` co `source_id` prefix `agentdata:` tai thoi diem s170-clarify	CON CHENH — Directus hom nay: 593 (tang them 17 records tu luc bao cao)
13	Chenh lech Directus - AD	576 - 563 = 13 records chi co trong Directus, KHONG co trong AD. Legacy records tu truoc khi co sync	DUNG — day la Directus-only records. Hom nay chenh lech: 593 - 565 = 28

So lieu cap nhat hom nay:

AD knowledge: 565 (tang 2 tu 563)
Directus knowledge_documents: 593 (tang 17 tu 576)
Chenh lech: 28 (tang tu 13 — Directus tang nhanh hon AD)

Nguyen nhan chenh lech 28: Directus co records tu:

Legacy data truoc khi co sync (13 records cu)
Directus flows tao records khong qua AD (15 records moi)

DOT-317 can scan va clean Directus-only orphans.

Clarify-2 Cau 4 — Ra Kien Truc Nuxt Theo HP

4a. Mo Hinh Data Flow Nuxt Hien Tai

                        User Browser
                             |
                        Nuxt SSR Pages
                        /pages/**/*.vue
                             |
              +--------------+--------------+
              |              |              |
         (A) Agent Data   (B) Directus   (B) Server Routes
         useAgentData()   $directus SDK  /server/api/registry/*
              |              |              |
         AD API (VPS)    Directus API    Directus API
         /api/kb/*       /items/*        /items/*
              |              |              |
            PG + Qdrant      PG             PG

Phan loai:

(A) Nuxt → Agent Data API → PG/Qdrant: useAgentData.ts, agentDataClient.ts → HP-COMPLIANT. AD chi tra ve IDs, content fetch qua Directus.
(B) Nuxt → Directus Items API → PG: Toan bo /server/api/registry/*.ts (14 files)
- Directus SDK trong pages/composables → HP-COMPLIANT. Directus la Tier 1 infrastructure (HP Dieu 5).
(C) Nuxt → PG truc tiep: 0 files. KHONG vi pham.
(D) Nuxt → hardcode SQL: 0 files. KHONG vi pham.

4b. Liet Ke Pages Theo Nhom

Nhom B — Nuxt → Directus API (HP-compliant):

#	Page Path	Data Source	Key API Call
1	`/pages/knowledge/registries/index.vue`	Directus SDK + `/api/registry/*`	`readItems('meta_catalog')`
2	`/pages/knowledge/registries/all/index.vue`	Directus SDK + `/api/registry/*`	`readItems('meta_catalog')`
3	`/pages/knowledge/registries/health/index.vue`	Server route	`$fetch('/api/registry/health')`
4	`/pages/knowledge/registries/matrix/index.vue`	Server route	`$fetch('/api/registry/matrix')`
5	`/pages/knowledge/registries/species/index.vue`	Server route	`$fetch('/api/registry/species-matrix')`
6	`/pages/knowledge/registries/taxonomy/index.vue`	Directus SDK	`readItems(...)`
7	`/pages/knowledge/registries/taxonomy/[facetCode]/[level].vue`	Directus SDK	`readItems(...)`
8	`/pages/knowledge/registries/changelog/index.vue`	Directus SDK	`readItems(...)`
9	`/pages/knowledge/registries/unmanaged/index.vue`	Server route	`$fetch('/api/registry/unmanaged')`
10	`/pages/knowledge/registries/system_issue/index.vue`	Server route	`$fetch('/api/registry/system-issues')`
11	`/pages/knowledge/registries/system_issue/[issue_class].vue`	Server route	`$fetch('/api/registry/system-issues-subgroups')`
12	`/pages/knowledge/registries/system_issue/[issue_class]/[sub_class].vue`	Server route	`$fetch('/api/registry/system-issues-detail')`
13	`/pages/knowledge/registries/[entityType]/index.vue`	Directus SDK	`readItems(...)`
14	`/pages/knowledge/registries/[entityType]/[id].vue`	Directus SDK	`readItems(...)`

Nhom C/D — Vi pham HP: 0 files.

Server routes (14 files) — tat ca dung Directus API:

#	Route	Directus Collection	Pattern
1	`composition.get.ts`	`entity_species`, `species_collection_map`	`$fetch(directusUrl/items/...)`
2	`counts.get.ts`	`meta_catalog`	`$fetch(directusUrl/items/...)`
3	`health.get.ts`	`species_collection_map`, `collection_registry`, `birth_registry`	`$fetch(directusUrl/items/...)`
4	`matrix.get.ts`	`meta_catalog` + dynamic collections	`$fetch(directusUrl/items/...)`
5	`pivot-query.get.ts`	`pivot_definitions`, `pivot_results`	`$fetch(directusUrl/items/...)`
6	`raw-counts.get.ts`	`meta_catalog`, `system_issues`	`$fetch(directusUrl/items/...)`
7	`refresh-counts.post.ts`	`meta_catalog` + dynamic collections	`$fetch(directusUrl/items/...)`
8	`species-matrix.get.ts`	`entity_species`	`$fetch(directusUrl/items/...)`
9	`species-summary.get.ts`	species collections	`$fetch(directusUrl/items/...)`
10	`system-issues.get.ts`	`system_issues`	`$fetch(directusUrl/items/...)`
11	`system-issues-groups.get.ts`	`system_issues`	`$fetch(directusUrl/items/...)`
12	`system-issues-subgroups.get.ts`	`system_issues`	`$fetch(directusUrl/items/...)`
13	`system-issues-detail.get.ts`	`system_issues`	`$fetch(directusUrl/items/...)`
14	`unmanaged.get.ts`	`collection_registry`	`$fetch(directusUrl/items/...)`

4c. /registries Cu The

Path chinh: /pages/knowledge/registries/ (14 .vue files) Nhom: B — Directus API (HP-compliant) Data source: Directus SDK (readItems()) hoac Nuxt server routes (/api/registry/*) → Directus Items API Hardcode __ hoac /: KHONG. Grep 0 matches cho knowledge__, docs__, _fs_key trong toan bo Nuxt source. PA5 co lam vo khong: KHONG. Registries pages doc tu Directus collections (meta_catalog, entity_species, system_issues, etc.) — KHONG phai tu kb_documents.key. PA5 migrate kb_documents.key khong anh huong registries.

4d. Impact PA5

Pages vo sau PA5: KHONG CO PAGE NAO VO.

Ly do:

Registries pages doc tu Directus collections (meta_catalog, entity_species, etc.) — KHONG doc tu kb_documents
Cac page dung useAgentData() nhan document_id format / tu AD API — AD API da convert __→/ truoc khi tra ve. Sau PA5, PG key = / = doc_id, nen API van tra ve cung gia tri
Knowledge pages (doc list, doc detail) dung Directus knowledge_documents.source_id format agentdata:knowledge/... — cung khong chua __

Deprecation order: Khong can deprecate page nao. PA5 la transparent migration.

4e. Ap Tuyen Ngon 3 Cau Cho Nuxt

Vinh vien hay tam? Phat hien hien tai: Nuxt registries KHONG vi pham HP (tat ca la nhom B). Nhung viec xac nhan nay la TAM (grep 1 lan). Vinh vien = can co che tu dong kiem tra lien tuc → dan toi Cau 6 (DOT-319).

Con co hoi nham khong? CO. Khong co gi chan dev moi viet page truc tiep query PG. Nuxt khong co lint rule, pre-commit hook, hay CI check ngan import pg, knex, hay hardcode SQL. Co hoi nham = 100% cho developer moi.

Loi = co hoi vang — pattern gi sau hon? Pattern: He thong KHONG co co che tu phat hien HP violation (NT-05 bi vi pham o cap he thong). Report S170-FINALIZE claim "Nuxt CLEAN" chi dua vao grep keyword — day la point-in-time check, KHONG PHAI co che vinh vien. Bat ky luc nao dev co the them file moi vi pham HP ma khong ai biet cho den khi production gap van de.

→ Dan den Cau 6: DOT-319 HP Compliance Scanner.

HP Violations Other Layers (Out-of-scope, ghi nhan cho S172)

Trong qua trinh ra soat, KHONG phat hien HP violation o Nuxt layer. Cac layer khac (Directus flows, API endpoints) chua duoc scan trong phien nay — can ra soat rieng.

Clarify-2 Cau 5 — DOT-318 Partial Index DDL Cu The

DDL Day Du

-- Partial index: chi index rows co literal __ trong key
-- Dung cho DOT-318 encoding drift detector
CREATE INDEX CONCURRENTLY idx_kb_documents_key_double_underscore
ON kb_documents (key)
WHERE key LIKE '%\_\_%' ESCAPE '\';

Luu y:

CONCURRENTLY = khong lock bang khi tao index
ESCAPE '\' = dam bao _ la literal, khong phai LIKE wildcard
Partial index chi chua rows vi pham → sau PA5 migrate, index = EMPTY (0 rows)

EXPLAIN Gia Dinh 1M Rows

Truoc PA5 (gia su ~900 rows vi pham trong 1M):

-- Query: SELECT count(*) FROM kb_documents WHERE key LIKE '%\_\_%' ESCAPE '\'

-- KHONG co partial index:
Seq Scan on kb_documents  (cost=0.00..25000.00 rows=1000000)
  Filter: (key LIKE '%\_\_%' ESCAPE '\')
  Rows Removed by Filter: 999100
  Estimated time: 2-5 seconds (full table scan 1M rows)

-- CO partial index:
Index Only Scan using idx_kb_documents_key_double_underscore
  (cost=0.00..5.00 rows=900)
  Estimated time: < 1ms (chi scan 900 entries trong index)

Sau PA5 migrate (0 rows vi pham trong 1M):

-- CO partial index (index EMPTY):
Index Only Scan using idx_kb_documents_key_double_underscore
  (cost=0.00..1.00 rows=0)
  Estimated time: < 0.1ms (index empty, return 0 ngay)

Ket hop voi CHECK constraint

-- PA5 Step 2: CHECK constraint ngan __ moi
ALTER TABLE kb_documents 
ADD CONSTRAINT chk_key_no_double_underscore 
CHECK (key NOT LIKE '%\_\_%' ESCAPE '\');

-- Partial index + CHECK = 2 lop bao ve:
-- 1. CHECK: ngan INSERT/UPDATE voi __ (enforcement)
-- 2. Partial index: DOT-318 scan nhanh < 1ms (detection)
-- Sau PA5: ca 2 deu = 0 violations. Redundant nhung defense-in-depth.

Clarify-2 Cau 6 — TAN GOC: DOT-319 HP Compliance Scanner

Goc Re

NT-05 (tu phat hien, tu sua) bi vi pham o cap he thong. He thong KHONG co co che tu dong phat hien HP violation. Ket qua:

S170-FINALIZE claim "Nuxt CLEAN" chi dua vao grep keyword 1 lan
Dev moi co the viet page truc tiep query PG bat ky luc nao
Khong co CI check, khong co daily scan, khong co alert
Vi pham chi bi phat hien khi con nguoi manually review

Fix 1 instance (xoa 1 page) = chua ngon. Fix vinh vien = co che tu phat hien.

6.1 Thiet Ke Ky Thuat

Bang hp_compliance_rules:

CREATE TABLE hp_compliance_rules (
    rule_id       TEXT PRIMARY KEY,
    layer         TEXT NOT NULL CHECK (layer IN ('nuxt','directus','api','pg')),
    scan_target   TEXT NOT NULL,           -- e.g. 'web/**/*.{vue,ts}'
    forbidden_pattern TEXT NOT NULL,       -- regex pattern
    severity      TEXT NOT NULL CHECK (severity IN ('critical','high','medium','low')),
    hp_article_ref TEXT NOT NULL,          -- e.g. 'HP-CS-01', 'Dieu 5'
    description   TEXT,
    enabled       BOOLEAN DEFAULT true,
    created_at    TIMESTAMPTZ DEFAULT now()
);

Bang hp_violations:

CREATE TABLE hp_violations (
    violation_id  SERIAL PRIMARY KEY,
    scan_id       TEXT NOT NULL,           -- UUID cua lan scan
    rule_id       TEXT NOT NULL REFERENCES hp_compliance_rules(rule_id),
    file_path     TEXT NOT NULL,
    line_number   INTEGER,
    snippet       TEXT,                    -- dong code vi pham (max 200 chars)
    status        TEXT DEFAULT 'open' CHECK (status IN ('open','acknowledged','fixed','false_positive')),
    detected_at   TIMESTAMPTZ DEFAULT now(),
    resolved_at   TIMESTAMPTZ
);

Seed rules ban dau:

INSERT INTO hp_compliance_rules VALUES
-- Nuxt layer: cam truc tiep PG
('HP-NUXT-001', 'nuxt', 'web/**/*.{vue,ts,js}', 
 'import.*pg|require.*pg|createPool|new Pool|knex|prisma|sequelize|drizzle',
 'critical', 'HP Dieu 5 + NT-06', 'Nuxt KHONG duoc import PG client'),

-- Nuxt layer: cam hardcode SQL
('HP-NUXT-002', 'nuxt', 'web/**/*.{vue,ts,js}',
 'SELECT\s+.*FROM\s+|INSERT\s+INTO|UPDATE\s+.*SET|DELETE\s+FROM',
 'critical', 'HP Dieu 5 + NT-06', 'Nuxt KHONG duoc chua SQL statement'),

-- Nuxt layer: cam hardcode __ encoding
('HP-NUXT-003', 'nuxt', 'web/**/*.{vue,ts,js}',
 'knowledge__|docs__|_fs_key|__.*doc_id',
 'high', 'HP CP-02', 'Nuxt KHONG duoc hardcode __ encoding'),

-- API layer: cam bypass Directus
('HP-API-001', 'api', 'agent_data/**/*.py',
 'psycopg2\.connect|pg8000|asyncpg\.connect',
 'medium', 'HP Dieu 5', 'API chi duoc dung pg_store module, khong direct connect'),

-- PG layer: detect encoding drift
('HP-PG-001', 'pg', 'kb_documents.key',
 '__',
 'high', 'HP CP-02 + PA5', 'PG key khong duoc chua __ sau PA5');

6.2 Quy Trinh

QT-HP-01: Scan Dinh Ky (Daily)

Trigger: Cron daily 03:00 UTC
For each rule WHERE enabled = true AND layer IN ('nuxt','api'):
- rg --pcre2 -n "{forbidden_pattern}" {scan_target} (exclude node_modules, .nuxt)
- Moi match → INSERT INTO hp_violations
For each rule WHERE layer = 'pg':
- SELECT count(*) FROM {table} WHERE {column} LIKE '%{pattern}%'
- If count > 0 → INSERT INTO hp_violations
Output: scan report → DOT status update

QT-HP-02: Scan Event-Driven (Git Post-Commit)

Trigger: git post-commit hook
Chi scan files trong commit (git diff --name-only HEAD~1)
Apply rules cho changed files only → fast (< 5 sec)
Vi pham → WARNING trong terminal (khong block commit de tranh friction)

QT-HP-03: Quarantine + Block Deploy

If hp_violations WHERE severity = 'critical' AND status = 'open' > 0:
- Block deploy (CI step fail)
- Auto tao system_issue trong Directus
- Auto notify (webhook/email)
If severity = 'high': WARNING nhung khong block
If severity = 'medium'/'low': Log only

6.3 DOT Cap 2 Chieu (CP-12)

DOT-319A (chinh): HP Compliance Scanner

Scan Nuxt/API/PG source code va data theo hp_compliance_rules
Output: hp_violations table + DOT status
Tier: A (critical — ngan HP violation)
Trigger: cron daily + git post-commit (dual-trigger NT-07)

DOT-319B (phu): HP Rules Verifier

Scan hp_compliance_rules table
For each rule: verify hp_article_ref con ton tai trong HP document
- search_knowledge("constitution") → check article ref
- Neu HP thay doi (article renamed/removed) → flag rule as STALE
Output: rules health report
Tier: B (advisory)
Trigger: cron weekly + manual (khi HP cap nhat)

6.4 Contract Test (Dieu 31)

Test 1 — True Positive (Nuxt PG import):

# Tao file gia vi pham HP
echo 'import { Pool } from "pg"' > web/test-hp-violation.ts
# Chay scanner
dot-hp-scan --target web/test-hp-violation.ts
# Expected: 1 violation, rule HP-NUXT-001, severity critical
# Cleanup
rm web/test-hp-violation.ts

→ Expect phat hien trong < 5 phut (daily scan) hoac < 5 giay (post-commit).

Test 2 — True Negative (Nuxt hop le):

# Tao file hop le
echo 'const data = await $fetch("/api/registry/health")' > web/test-hp-clean.ts
# Chay scanner
dot-hp-scan --target web/test-hp-clean.ts
# Expected: 0 violations
# Cleanup
rm web/test-hp-clean.ts

→ Expect KHONG false positive.

Test 3 — PG Encoding Drift:

-- Trong test environment, insert row vi pham
INSERT INTO kb_documents (key, data) VALUES ('test__violation', '{}');
-- Chay scanner
-- Expected: 1 violation, rule HP-PG-001
-- Cleanup
DELETE FROM kb_documents WHERE key = 'test__violation';

Test 4 — Rule Staleness:

-- Them rule voi article ref khong ton tai
INSERT INTO hp_compliance_rules VALUES 
  ('TEST-001', 'nuxt', 'web/**', 'test', 'low', 'HP-FAKE-999', 'test');
-- Chay DOT-319B
-- Expected: flag TEST-001 as STALE
-- Cleanup
DELETE FROM hp_compliance_rules WHERE rule_id = 'TEST-001';

6.5 Auto 100%

Buoc	Tu dong?	Co che
Daily scan	CO	Cron 03:00 UTC
Post-commit scan	CO	Git hook
Phat hien vi pham	CO	ripgrep + SQL
Ghi hp_violations	CO	INSERT auto
Tao system_issue	CO	Directus flow trigger
Block deploy (critical)	CO	CI step check hp_violations
Notify	CO	Webhook
Update DOT status	CO	DOT API call
Review false positive	MANUAL	Human review

Buoc manual duy nhat: Review false positive de danh dau status = 'false_positive'. Day la THIET KE DUNG (human-in-the-loop cho false positive), KHONG PHAI thieu tu dong.

6.6 Scale (CQ-3)

Metric	Hom nay	10k files	Co che
Nuxt source files	581	10,000	ripgrep scan full: ~2 sec (10k files)
Daily full scan	~1 sec	~2 sec	ripgrep parallelized
Post-commit incremental	< 0.5 sec	< 0.5 sec	Chi scan changed files
PG scan (voi partial index)	< 1ms	< 1ms	Index only scan
hp_compliance_rules	5 rules	50 rules	PG table, < 1ms scan
hp_violations (1 year)	0	~1000	PG table, indexed by scan_id

Incremental scan cho post-commit:

# Chi scan files thay doi
CHANGED_FILES=$(git diff --name-only HEAD~1 | grep -E '\.(vue|ts|js)$')
for f in $CHANGED_FILES; do
    rg --pcre2 -n "$PATTERN" "$f"  # Per-rule scan
done

Clarify-2 — Self-Check

#	Cau	Deliverable	Status
1	Dut diem so rows	Bang 5 dong + con so tu API (SSH khong kha dung)	✅ DONE (caveat: can SSH verify PG key format)
2	Manual step → 0	Script regex replace, 0 buoc tay	✅ DONE
3	563/576/13	Bang 3 dong + cap nhat hom nay	✅ DONE
4	Ra kien truc Nuxt	4a-e day du, 14 pages + 14 routes phan loai	✅ DONE — Nuxt = Nhom B (Directus API), 0 vi pham C/D
5	DDL partial index	DDL + EXPLAIN + CHECK constraint	✅ DONE
6	DOT-319 design	6.1-6.6 day du, 5 rules seed, 4 contract tests, scale CQ-3	✅ DONE

Tracker Updates

Tracker	Status	Ghi chu
TD-NUXT-HP-VIOLATION-REGISTRIES	KHONG AP DUNG	Registries = Nhom B (Directus API), KHONG vi pham HP. Phat hien S170-FINALIZE claim "CLEAN" la DUNG cho registries, nhung co che verify la TAM (1-time grep)
TD-DOT-319-HP-SCANNER	PROPOSED	Design day du trong Cau 6. Cho Desktop duyet
TD-DOC-ID-ENCODING	HOLD	Can SSH verify PG key `__` count truoc khi proceed PA5 migration

S170-CLARIFY-2 hoan tat. 6/6 cau co deliverable. CHO Desktop duyet. Luu y: SSH VPS port 22 connection refused — can khoi phuc truoc khi chay PA5 migration.

S170-INFRA-HEALTH — SSH Restore + Cau 1 Dut Diem + Monitoring Audit

Date: 2026-04-07 | Phien: S170-INFRA-HEALTH | Agent: Claude Code (Opus 4.6) Constraint: READ-ONLY. Chi restore SSH. KHONG tu cai monitoring.

IH-1: SSH Restore + Root Cause

Ket qua

SSH da KHOI PHUC — khong can sua gi. Port 22 tu dong kha dung tro lai.

$ ssh contabo "echo ok; uptime"
ok
up 53 days, 19:02,  2 users,  load average: 0.29, 0.31, 0.43

Root Cause Analysis

Yeu to	Trang thai	Loai tru?
fail2ban	Active. 9 IPs banned. IP cua Huyen KHONG trong banned list	✅ Loai tru
UFW firewall	Active. OpenSSH ALLOW Anywhere. Khong thay doi	✅ Loai tru
sshd service	KHONG restart trong 7 ngay gan nhat. Uptime 53 ngay	✅ Loai tru
conntrack table	19/262144 entries. KHONG full	✅ Loai tru
sshd_config	`PermitRootLogin prohibit-password`, `PasswordAuthentication no` — key-only. Khong thay doi	✅ Loai tru
Kernel/network drops	Chi thay UFW BLOCK cho random port scanners, KHONG thay block port 22	✅ Loai tru

Ket luan: Transient network issue — VPS khong block SSH. Root cause o tang network giua macOS va Contabo (ISP routing, hop trung gian, hoac Contabo infra). Evidence:

Lan 1: connection refused (contabo alias) — TCP RST tai hop trung gian
Lan 2: Permission denied then timeout (nmhuyen@) — TCP bi drop/timeout
Lan 3 (30 phut sau): SSH hoat dong binh thuong, khong can thay doi gi

KHONG CHAC 100%: Co the fail2ban da ban IP cua Huyen roi tu unban (bantime het). Nhung kiem tra journal fail2ban 2h gan nhat khong thay ban/unban IP nao cua Viet Nam.

Pattern NT-05: SSH down ma khong ai biet

Bat ke root cause la gi, van de THUC la: SSH bi mat ket noi ~30 phut va KHONG CO alert nao den Huyen. Uptime Kuma co 4 monitors nhung KHONG co SSH monitor va KHONG co notification channel cau hinh. → Chuyen sang IH-3.

IH-2: Cau 1 Dut Diem — So Rows Migration PA5

PG Query Truc Tiep (2026-04-07, SSH verified)

-- kb_documents
SELECT COUNT(*) FROM kb_documents;                                    -- 1809
SELECT COUNT(*) FROM kb_documents WHERE key LIKE '%\_\_%' ESCAPE '\'; -- 1630
SELECT COUNT(*) FROM kb_documents WHERE key NOT LIKE '%\_\_%' ESCAPE '\'; -- 179
SELECT COUNT(*) FROM kb_documents WHERE key LIKE '%/%';               -- 0
SELECT COUNT(DISTINCT key) FROM kb_documents;                         -- 1809

-- kb_documents_history
SELECT COUNT(*) FROM kb_documents_history;                            -- 2473
SELECT COUNT(*) FROM kb_documents_history 
  WHERE document_key LIKE '%\_\_%' ESCAPE '\';                        -- 2473

-- kb_audit_log
SELECT COUNT(*) FROM kb_audit_log;                                    -- 3047
SELECT COUNT(*) FROM kb_audit_log 
  WHERE document_key LIKE '%\_\_%' ESCAPE '\';                        -- 3044

Sample keys (5 rows)

(empty string)
knowledge__current-state__reports__s109-cp2-cp3-report.md
knowledge__current-state__reports__s116-gpt-review-count-definitive-plan-2026-03-13.md
knowledge__dev__architecture__birth-registry-law.md
test__verify-cp23

Format: 100% keys dung __, 0 keys dung / trong PG. API convert __→/ khi tra ve.

179 keys KHONG co `__`

Day la flat keys khong co path separator: directus-100, directus-101, constitution-v1.11e, mission-count-verify-report, va 1 empty string. KHONG can migrate (khong co __ de convert).

Bang Dut Diem 4 Con So

Con so	Gia tri	Nguon	Dung/Sai	Giai thich
900	Health API `document_count`	S170-FINALIZE A3, API /health	⚠️ SAI NGUU CANH	Health API dem subset (Python logic, co the chi dem `knowledge/` prefix hoac active docs). PG thuc te co 1809 rows
7315	Tong 3 bang	S170-deep investigation	✅ GAN DUNG	Tai thoi diem do: ~1800 + ~2470 + ~3040 ≈ 7310. Hom nay: 1809 + 2473 + 3047 = 7329
4400	Khong ro nguon	S170-clarify (claim)	❌ KHONG XAC NHAN	Khong tim duoc query hoac context tao ra con so nay. Co the la estimate cu hoac tinh sai
1806	`key LIKE '%__%'` UNESCAPED	S170-finalize A2	⚠️ SAI KY THUAT	`_` trong LIKE la wildcard = 1 ky tu bat ky. `%__%` = string co ≥ 2 ky tu = gan nhu tat ca rows. Tai thoi diem do: 1809 total - 3 key ngan = 1806. Day KHONG PHAI so keys co literal `__`

Con So Dung Cho PA5 Migration

Bang	Total rows	Rows co literal `__`	Rows can migrate
`kb_documents`	1809	1630	1630 (UPDATE key = replace)
`kb_documents_history`	2473	2473	2473
`kb_audit_log`	3047	3044	3044
TONG	7329	7147	7147

PA5 Migration SQL (cap nhat voi so thuc)

BEGIN;
-- 1a. kb_documents: 1630 rows
UPDATE kb_documents SET key = replace(key, '__', '/') 
WHERE key LIKE '%\_\_%' ESCAPE '\';

-- 1b. kb_documents_history: 2473 rows
UPDATE kb_documents_history SET document_key = replace(document_key, '__', '/') 
WHERE document_key LIKE '%\_\_%' ESCAPE '\';

-- 1c. kb_audit_log: 3044 rows
UPDATE kb_audit_log SET document_key = replace(document_key, '__', '/') 
WHERE document_key LIKE '%\_\_%' ESCAPE '\';
COMMIT;

-- Step 2: CHECK constraint
ALTER TABLE kb_documents 
ADD CONSTRAINT chk_key_no_double_underscore 
CHECK (key NOT LIKE '%\_\_%' ESCAPE '\');

Sai Lech Health API vs PG

Source	Count	Ghi chu
PG `kb_documents` total	1809	SSOT — so rows thuc trong bang
Health API `document_count`	900	Dem o Python layer, co the chi dem "knowledge" docs hoac active docs
AD API `kb/list` items	900	Trung voi Health — cung Python logic
AD API `knowledge/` prefix	565	Subset cua 900
Directus `knowledge_documents`	593	Directus mirror, nhieu hon AD 28 records

Root cause sai lech: Health API va kb/list dung Python code (server.py) de dem, co the filter theo prefix hoac status. PG co 1809 rows bao gom ca test, flat keys, va non-knowledge docs. Con so PA5 phai dua tren PG truc tiep (1630), KHONG dua tren API (900).

TD-DOC-ID-ENCODING Update

Field	Gia tri cu	Gia tri moi
Status	HOLD (can SSH)	UNBLOCKED — SSH restored, PG verified
PA5 migration scope	~1806 rows (claim)	1630 rows kb_documents + 5517 rows history/audit = 7147 total
Blocker	SSH refused	Khong con blocker ky thuat. Cho Desktop duyet PA5

IH-3: Audit Monitoring Ha Tang

3a. Mo Hinh Monitoring Hien Tai (BEFORE)

┌─────────────────────────────────────────────────┐
│            VPS Monitoring Landscape              │
├─────────────────────────────────────────────────┤
│                                                  │
│  UPTIME KUMA (uptime-kuma container, port 3001)  │
│  ┌─────────────────────────────────────────┐     │
│  │ 4 Monitors:                             │     │
│  │  1. MCP Endpoint (keyword, 60s)    ✅   │     │
│  │  2. Agent Data Health (HTTP, 60s)  ✅   │     │
│  │  3. Directus Health (HTTP, 60s)    ✅   │     │
│  │  4. OPS Proxy (HTTP, 300s)         ✅   │     │
│  │                                         │     │
│  │ ⚠️ NOTIFICATION CHANNELS: 0 (NONE!)    │     │
│  │ → Monitors chay nhung KHONG AI NHAN    │     │
│  │   alert khi service down               │     │
│  └─────────────────────────────────────────┘     │
│                                                  │
│  CRON JOBS (41+ entries, root crontab)           │
│  ┌─────────────────────────────────────────┐     │
│  │ Category         | Count | Alert?       │     │
│  │ DOT scanners     | 12    | Log only ❌  │     │
│  │ Pivot/matrix     | 4     | Log only ❌  │     │
│  │ Integrity/D31    | 3     | Log only ❌  │     │
│  │ NRM/Governance   | 5     | Log only ❌  │     │
│  │ Backup           | 3     | Log only ❌  │     │
│  │ Reconcile        | 2     | Log only ❌  │     │
│  │ Disk monitor     | 1     | Log only ❌  │     │
│  │ MCP connectivity | 1     | Log only ❌  │     │
│  │ Other            | 10+   | Log only ❌  │     │
│  │                                         │     │
│  │ ⚠️ TAT CA ghi log file, KHONG gui alert│     │
│  │ → Cron fail im lang. S167 pattern lap    │     │
│  └─────────────────────────────────────────┘     │
│                                                  │
│  KHONG DUOC MONITOR:                             │
│  ❌ SSH (port 22)                                │
│  ❌ Nuxt app health                              │
│  ❌ Qdrant health                                │
│  ❌ PostgreSQL health                            │
│  ❌ Docker container restarts                    │
│  ❌ Memory / CPU                                 │
│  ❌ Cron job execution (co chay khong?)          │
│  ❌ Disk approaching threshold                   │
│  ❌ SSL cert expiry                              │
│  ❌ fail2ban alerts (10K+ bans, ai biet?)        │
│                                                  │
│  S167 PATTERN LAP LAI:                           │
│  "Nguoi quet rac nghi viec tu khi chuyen VPS,    │
│   khong ai biet vi khong co monitoring"           │
│  → SSH down 30 phut, khong ai biet               │
│  → Cron fail im lang, khong ai biet              │
│  → Next incident: ???                             │
└─────────────────────────────────────────────────┘

Gap Analysis

Service/Resource	Duoc monitor?	Alert den Huyen?	Gap
API (MCP, Health)	✅ Uptime Kuma	❌ No notification	CRITICAL: monitor co nhung alert KHONG
Directus	✅ Uptime Kuma	❌ No notification	CRITICAL
SSH	❌	❌	CRITICAL: hom nay da bi
Nuxt	❌	❌	HIGH
PostgreSQL	❌	❌	HIGH
Qdrant	❌	❌	HIGH
Docker containers	❌	❌	HIGH
Cron jobs alive	❌	❌	CRITICAL: S167 pattern
Disk space	✅ Script	❌ Log only	MEDIUM
Memory/CPU	❌	❌	MEDIUM
SSL cert	✅ certbot timer	❌ No alert if fail	MEDIUM
fail2ban	✅ Active	❌ No alert	LOW

Ket luan: Uptime Kuma CHAY nhung VO DUNG vi 0 notification channel. 41+ cron jobs CHAY nhung TAT CA chi log, KHONG alert. He thong KHONG CO kha nang tu phat hien (vi pham NT-05 o cap ha tang).

3b. PG FIRST Analysis (NT-13)

pg_cron da cai chua? KHONG. Query pg_extension WHERE extname = 'pg_cron' tra ve 0 rows.

PG co lam monitoring cho VPS duoc khong?

Kha nang	PG native	pg_cron	Ghi chu
Self-health (connections, locks, bloat)	✅ pg_stat_* views	✅ scheduled queries	PG co the tu monitor chinh no
Disk usage	❌	❌	PG khong thay disk ngoai data dir
SSH port check	❌	❌	PG khong co network probe
Docker container status	❌	❌	PG khong thay Docker
Cron job liveness	❌	Partial — ghi heartbeat vao bang	PG co the nhan heartbeat tu cron
External service (Nuxt, Qdrant)	❌	❌	PG khong co HTTP client
Alert/notification	❌	❌	PG khong gui email/webhook native

Ket luan PG FIRST: PG + pg_cron co the:

Tu monitor chinh no (connections, replication, bloat) ✅
Nhan heartbeat tu cron jobs (INSERT vao bang, pg_cron check staleness) ✅
KHONG THE monitor SSH, Docker, disk, external services, GUI alert ❌

PG KHONG DU cho VPS monitoring. Can tool bo sung.

Tool toi thieu

Uptime Kuma DA CO va da chay. Khong can cai tool moi. Chi can:

Cau hinh notification channel (Telegram bot / email SMTP / webhook)
Them monitors cho SSH, Nuxt, Qdrant, PG, cron heartbeat
pg_cron cho PG self-monitoring (optional, nice-to-have)

3c. DOT Cap VPS Health (5 Cau Section XII)

XII-1: Thiet Ke Ky Thuat

Khong can tao bang moi. Uptime Kuma da co database. Chi can cau hinh:

Uptime Kuma monitors can them:

#	Monitor Name	Type	Target	Interval	Severity
5	SSH Port	TCP	38.242.240.89:22	60s	CRITICAL
6	Nuxt Health	HTTP	https://vps.incomexsaigoncorp.vn/	60s	CRITICAL
7	PostgreSQL	Docker (exec)	`pg_isready -U incomex`	60s	CRITICAL
8	Qdrant Health	HTTP	http://qdrant:6333/healthz (internal)	60s	HIGH
9	Cron Heartbeat	HTTP keyword	File mtime check or heartbeat endpoint	300s	HIGH
10	Disk Usage	HTTP keyword	Script endpoint or push monitor	300s	MEDIUM
11	Docker Containers	Docker	Container count = expected count	120s	HIGH

Notification channel:

Channel	Tool	Recipient	Khi nao
Telegram Bot	Uptime Kuma native	@nmhuyen (hoac group)	Moi CRITICAL/HIGH down
Email SMTP	Uptime Kuma native	huyen@incomexsaigoncorp.vn	Daily digest + CRITICAL

Cron heartbeat pattern (PG-assisted):

-- Optional: bang heartbeat trong PG
CREATE TABLE IF NOT EXISTS cron_heartbeat (
    job_name    TEXT PRIMARY KEY,
    last_run    TIMESTAMPTZ NOT NULL DEFAULT now(),
    status      TEXT DEFAULT 'ok',
    duration_ms INTEGER
);

-- Moi cron job INSERT/UPDATE cuoi script:
-- docker exec postgres psql -U incomex -d incomex_metadata \
--   -c "INSERT INTO cron_heartbeat VALUES ('dot-kb-verify', now(), 'ok', 1234)
--       ON CONFLICT (job_name) DO UPDATE SET last_run = now(), status = 'ok', duration_ms = 1234;"

-- Uptime Kuma monitor: check heartbeat freshness
-- SELECT COUNT(*) FROM cron_heartbeat WHERE last_run < now() - interval '26 hours';
-- Expected: 0

XII-2: Quy Trinh

QT Code	Mo ta	Trigger	Output
QT-VPS-01	Scan dinh ky	Uptime Kuma 60s intervals	Monitor status GREEN/RED
QT-VPS-02	Alert	Uptime Kuma notification	Telegram message + email
QT-VPS-03	Verify alert channel	Weekly test ping	Confirm Telegram/email still nhan duoc

XII-3: DOT Cap 2 Chieu (CP-12) + Dual-Trigger (NT-07)

DOT-VPS-HEALTH-A (chinh): VPS Service Monitor

Tool: Uptime Kuma (da co)
Monitors: SSH, HTTP services, PG, Docker, cron heartbeat
Trigger 1 (cron): Uptime Kuma internal polling 60s
Trigger 2 (event): Docker healthcheck events / systemd notify
Alert: Telegram + email khi service down > 2 phut

DOT-VPS-HEALTH-B (phu): Alert Channel Verifier

Tool: Cron script hoac Uptime Kuma "Push" monitor
Kiem tra: Telegram bot con hoat dong? Email SMTP con gui duoc?
Trigger: Weekly cron (Sunday 00:00)
Test: Gui test message → verify delivery
Neu fail → ghi system_issue severity HIGH

Dual-trigger:

Trigger 1: Uptime Kuma polling lien tuc (60s) = cron-equivalent
Trigger 2: Docker healthcheck + systemd on-failure = event-driven
Ca 2 triggers doc lap, 1 fail thi con lai van phat hien

XII-4: Quan Tri Thay Doi

Khi them service moi (e.g., them Redis container):

Them monitor trong Uptime Kuma (UI hoac API)
Them cron heartbeat dong trong script cua service
DOT-VPS-HEALTH-B tu dong verify ca monitor moi

Tu dang ky: Moi Docker container co healthcheck → Uptime Kuma Docker monitor tu phat hien container moi qua Docker API.

XII-5: Auto 100%

Buoc	Tu dong?	Co che
Detect service down	✅	Uptime Kuma polling 60s
Alert Huyen	✅	Telegram bot instant
Log event	✅	Uptime Kuma history
Verify alert channel	✅	Weekly test ping cron
Add new service monitor	⚠️ SEMI	Docker auto-discover, nhung custom monitors can manual add
Restart failed service	❌ KHONG	Docker restart policy da co. KHONG auto-restart ngoai Docker

Buoc manual: Them custom monitor khi them service KHONG phai Docker container. Day la by design — khong auto-discover non-Docker services.

3d. Scale (CQ-3)

Metric	Hom nay	N VPS tuong lai	Co che
VPS count	1	N	Uptime Kuma ho tro multi-host native. Them remote monitors chi IP:port
Monitors	4 → 11	11 × N	Uptime Kuma handle 100+ monitors thoai mai
Notification	0 → 2 channels	Same	1 Telegram group cho tat ca VPS
Storage	SQLite ~5MB	~50MB (1 year, N VPS)	Uptime Kuma tu cleanup old data

Design scale: Uptime Kuma la centralized monitor. Them VPS moi = them monitors (remote TCP/HTTP). Khong can cai them tool.

Gioi han: Neu > 10 VPS, nen chuyen sang Prometheus + Grafana + Alertmanager. Nhung hien tai 1 VPS, Uptime Kuma la du.

3e. Dut Khoat: Thoi Gian Phat Hien + Kenh Alert

Hien tai (BEFORE):

SSH down → KHONG AI BIET cho den khi developer thu SSH va gap loi
Thoi gian phat hien: vo han (tu vai phut den vai ngay)
Kenh alert: KHONG CO

Sau khi cau hinh DOT-VPS-HEALTH (AFTER):

Service down	Phat hien trong	Kenh alert	Den Huyen
SSH	< 2 phut	Telegram Bot	✅ Push notification dien thoai
API/MCP	< 2 phut	Telegram Bot	✅
Directus	< 2 phut	Telegram Bot	✅
Nuxt	< 2 phut	Telegram Bot	✅
PostgreSQL	< 2 phut	Telegram Bot	✅
Cron job chet	< 26 gio	Telegram Bot	✅ (watchdog pattern)
Disk > 85%	< 5 phut	Telegram Bot	✅
Alert channel chet	< 7 ngay	Weekly verify	✅ (fallback: email)

Cam ket: SSH (hoac bat ky critical service) down → Huyen nhan Telegram notification trong duoi 2 phut, 24/7, khong can ai dang online.

IH-4: HP Violations Other Layers (Out-of-scope)

Phat hien trong qua trinh audit:

Uptime Kuma 0 notification channel — monitor chay ma alert KHONG gui. Vi pham NT-05 (tu phat hien nhung KHONG tu thong bao).
41+ cron jobs log-only — tat ca ghi file, khong ai doc. S167 pattern lap lai.
Ghi nhan cho S172 hoac mission rieng.

IH-5: Self-Check

#	Muc	Status	Evidence
1	SSH up verified	✅	`ssh contabo "echo ok"` → OK, uptime 53 days
2	Root cause co evidence	✅	Transient network. fail2ban/UFW/sshd/conntrack loai tru. KHONG CHAC 100% nhung co evidence loai tru 5 nguyen nhan server-side
3	4 con so dut diem	✅	900 = API subset, 7315 ≈ 7329 (3 bang), 4400 = unverified, 1806 = unescaped LIKE. Dung: PG co 1809 rows, 1630 co `__`, migration = 7147 rows
4	Mo hinh monitoring BEFORE	✅	Uptime Kuma 4 monitors/0 notifications + 41 crons/0 alerts. Gap analysis 12 items
5	PG FIRST analysis	✅	pg_cron KHONG co. PG tu monitor chinh no duoc nhung KHONG du cho VPS. Uptime Kuma da co = tool du
6	DOT cap 5 cau XII + dual-trigger	✅	DOT-VPS-HEALTH-A (poll 60s) + DOT-VPS-HEALTH-B (verify weekly). Dual: Uptime Kuma poll + Docker healthcheck
7	Scale + alert channel cu the	✅	Scale N VPS = them remote monitors. Alert: Telegram Bot < 2 phut

Tracker Updates

Tracker	Status	Ghi chu
TD-DOC-ID-ENCODING	UNBLOCKED	SSH restored. PG verified: 1630 rows `__` can migrate. Cho Desktop duyet PA5
TD-INFRA-MONITORING-GAP	NEW — CRITICAL	Uptime Kuma 0 notification channels. 41 cron jobs 0 alerts. NT-05 vi pham o ha tang
TD-DOT-VPS-HEALTH	PROPOSED	Design day du. Cau hinh Uptime Kuma + Telegram Bot. Cho Desktop duyet

S170-INFRA-HEALTH hoan tat. 7/7 muc self-check tick. BLOCKER #1: Uptime Kuma can cau hinh notification NGAY (P0 — co tool, chi thieu config). Cho Desktop duyet: PA5 migration (7147 rows) + DOT-VPS-HEALTH (Uptime Kuma + Telegram).

S170-FINALIZE — 5 Muc Bat Buoc Dong PA5

Section A — 4 Cau So Lieu

A1: 15 hay 17 call sites?

A2: Mau thuan Q3 <-> PA2

A3: 561 A->B status

A4: Commit hash dau tien dung _fs_key

Section B — Bang 5 PA x 6 Cot (30 O)

Section C — 5 Cau OR XII Cho PA5

C1: DDL Cu The

C2: QT Dat Ten

C3: DOT Cap 2 Chieu (CP-12) — DESIGN MO RONG REGISTRY-DRIVEN

C4: Contract Test (Dieu 31) — 5 Test Case Cho 5 Tang

C5: Tu Dong 100%

Section D — Diagram BEFORE/AFTER (Du 5 Tang)

BEFORE (Hien Tai) — 5 Tang Co Lech

AFTER PA5 — 5 Tang Dong Bo Format /

Verify CP-02 Cho Tang 5 (Nuxt)

Section E — AP-CLOSE Checklist

E1: 5 Self-Check Verify Cua S170-CLARIFY

E2: Git Commit Hash

E3: Tracker TD-DOC-ID-ENCODING

E4: Self-Check 5 Muc Phien Nay

CLARIFY-2 — Lam Ro 5 Diem + Dao Tan Goc Lo Hong NT-05

Clarify-2 Cau 1 — DUT DIEM So Rows Migration

Query hom nay (2026-04-07)

Bang giai thich 4 con so mau thuan

Con so dut diem hom nay

Clarify-2 Cau 2 — Manual Step Phai Ve 0

Buoc manual tu Section C5

Ap tuyen ngon cau 4: First thought = AUTO

Clarify-2 Cau 3 — Giai Thich 563/576/13

Clarify-2 Cau 4 — Ra Kien Truc Nuxt Theo HP

4a. Mo Hinh Data Flow Nuxt Hien Tai

4b. Liet Ke Pages Theo Nhom

4c. /registries Cu The

4d. Impact PA5

4e. Ap Tuyen Ngon 3 Cau Cho Nuxt

HP Violations Other Layers (Out-of-scope, ghi nhan cho S172)

Clarify-2 Cau 5 — DOT-318 Partial Index DDL Cu The

DDL Day Du

EXPLAIN Gia Dinh 1M Rows

Ket hop voi CHECK constraint

Clarify-2 Cau 6 — TAN GOC: DOT-319 HP Compliance Scanner

Goc Re

6.1 Thiet Ke Ky Thuat

6.2 Quy Trinh

6.3 DOT Cap 2 Chieu (CP-12)

6.4 Contract Test (Dieu 31)

6.5 Auto 100%

6.6 Scale (CQ-3)

Clarify-2 — Self-Check

Tracker Updates

S170-INFRA-HEALTH — SSH Restore + Cau 1 Dut Diem + Monitoring Audit

IH-1: SSH Restore + Root Cause

Ket qua

Root Cause Analysis

Pattern NT-05: SSH down ma khong ai biet

IH-2: Cau 1 Dut Diem — So Rows Migration PA5

PG Query Truc Tiep (2026-04-07, SSH verified)

Sample keys (5 rows)

179 keys KHONG co __

Bang Dut Diem 4 Con So

Con So Dung Cho PA5 Migration

PA5 Migration SQL (cap nhat voi so thuc)

Sai Lech Health API vs PG

TD-DOC-ID-ENCODING Update

IH-3: Audit Monitoring Ha Tang

3a. Mo Hinh Monitoring Hien Tai (BEFORE)

Gap Analysis

3b. PG FIRST Analysis (NT-13)

Tool toi thieu

3c. DOT Cap VPS Health (5 Cau Section XII)

XII-1: Thiet Ke Ky Thuat

XII-2: Quy Trinh

XII-3: DOT Cap 2 Chieu (CP-12) + Dual-Trigger (NT-07)

XII-4: Quan Tri Thay Doi

XII-5: Auto 100%

3d. Scale (CQ-3)

3e. Dut Khoat: Thoi Gian Phat Hien + Kenh Alert

IH-4: HP Violations Other Layers (Out-of-scope)

A4: Commit hash dau tien dung `_fs_key`

AFTER PA5 — 5 Tang Dong Bo Format `/`

179 keys KHONG co `__`