S174-FIX-03: Archive Workflow Legacy + Gỡ Network Orphan (GCS-01 + X-01)

Date: 2026-04-08 Mission: S174-FIX-03 Status: COMPLETE — 5/5 verify GREEN + 5min stability PASS

---

Bước 1 — Điều tra

Critical finding

/opt/workflow/postgres/data là PG production data — volume mount trong docker-compose.yml lines 14-16. KHÔNG thể mv /opt/workflow toàn bộ. Chỉ archive từng file legacy riêng.

Dependencies tìm được

#	Path	Nội dung	Loại
1	root crontab	0 2 * * * /opt/workflow/postgres/backup.sh	Cron → comment out
2	docker-compose.yml:14-16	PG volume mounts từ /opt/workflow/postgres/	GIỮ NGUYÊN
3	/opt/workflow/.env	GCS_BUCKET_BACKUP=huyen1974-system-backups-shared	Archive
4	/opt/workflow/docker-compose.postgres.yml	workflow-postgres service	Archive
5	workflow_internal network	Containers=0, orphan	Remove

Backup files analysis

• Từ 2026-03-12 trở đi: tất cả 20 bytes (gzip rỗng) — workflow-postgres container đã bị xóa
• Last successful backup to GCS: 2026-03-11 (GCS_OK)
• Cron vẫn chạy mỗi đêm, sinh file rỗng + Error: No such container: workflow-postgres

---

Bước 2.0 — Fresh PG Backup lên GDrive

#	Verify	Result
1	Script exit code	0 (set -euo pipefail)
2	Local file	vps-backup-20260408_161305.tar.gz, 75MB, 2026-04-08 16:13
3	Gunzip test	exit 0
4	Contents	PG dump 36MB + Qdrant 66MB + configs + scripts
5	GDrive remote	vps-backup-20260408_161305.tar.gz = 74.207 MiB (77,811,588 bytes)
6	Kuma heartbeat	{"ok":true} — pg-backup-gdrive push accepted

Rollback file: gdrive-backup:vps-backup-20260408_161305.tar.gz

---

Bước 2.1 — Archive Legacy

Actions taken

#	Action	Detail
1	Cron	Commented out: # S174-FIX-03 archived workflow legacy
2	Compose down	docker compose -f docker-compose.postgres.yml down → removed workflow_internal network
3	Archive .env	/opt/workflow/.env → .env.retired.20260408
4	Archive compose	docker-compose.postgres.yml → .retired.20260408 (+ backup file)
5	Archive script	backup.sh → .retired.20260408 (+ .pre-s174 variant)
6	Archive log	backup.log → .retired.20260408
7	Archive backups dir	backups/ → backups.retired.20260408/ (12 files, all ≤458 bytes)

Preserved (NOT archived)

• /opt/workflow/postgres/data/ — PRODUCTION PG DATA, mounted by docker-compose.yml:14
• /opt/workflow/postgres/init/ — PG init scripts, mounted by docker-compose.yml:16

---

Verify 5/5

#	Check	Result
1	crontab -l | grep workflow	Only commented lines: # S174-FIX-03 archived workflow legacy
2	Active files in /opt/workflow (not .retired, not data/init)	NONE — 0 active legacy files
3	Retired files exist	7 .retired.20260408 files present
4	docker network ls | grep workflow_internal	GONE
5	grep workflow-postgres|GCS_BUCKET_BACKUP in production configs	CLEAN — 0 matches

5-Minute Stability Check

All 7 containers healthy after archive:

• postgres — Up 2 weeks (healthy), normal checkpoint logs
• incomex-directus — Up 10 days (healthy), serving requests normally
• incomex-nuxt — Up ~1 hour (healthy), pre-existing 403/mdi warnings only
• incomex-agent-data — Up 2 days (healthy), Qdrant+PG probes OK
• incomex-qdrant — Up 3 weeks (healthy), serving queries
• incomex-nginx — Up 3 weeks, routing traffic
• uptime-kuma — Up 5 hours (healthy)

No new errors from archive operation.

---

Rollback

Component	Rollback command
Cron	crontab -l > /tmp/c.txt && sed -i 's/^# \(0 2.*backup.sh\)/\1/' /tmp/c.txt && crontab /tmp/c.txt
Files	for f in /opt/workflow/*.retired.20260408 /opt/workflow/postgres/*.retired.20260408; do mv "$f" "${f%.retired.20260408}"; done
Network	docker compose -f /opt/workflow/docker-compose.postgres.yml up -d (recreates network)
PG data	Download gdrive-backup:vps-backup-20260408_161305.tar.gz, extract postgresql-directus.sql.gz, restore via docker exec -i postgres psql

---

Status: GCS-01 + X-01 CLOSED

GCS-01 (workflow backup chain → GCS retired): Cron disabled, backup script archived, GCS env archived. New backup system (backup-to-gdrive.sh + rclone) verified working.

X-01 (workflow_internal orphan network): Removed via docker compose down.