KB-1286

P9-G6 Backup Fix — Option A + D Execution Log (2026-04-27)

13 min read Revision 1
reportbackup-fixgate-a-dpg-backupdirectussandbox_tacgrantscript-hardening2026-04-27p9-g6

title: P9-G6 Backup Fix — Option A + D Execution Log date: 2026-04-27 date_utc: 2026-04-27T15:00Z gate: backup-fix-option-a-d executor: Claude Code (governed AI execution gate) authority_chain: GPT R20 fix path → GPT R20+R21+R22 review PASS w/ patches → User authorized → Claude Code execute effort: high verdict: PASS mutations:

  • id: A name: narrow GRANT target: PostgreSQL role directus on schema sandbox_tac status: PASS
  • id: D name: script hardening target: /opt/incomex/scripts/pg-backup.sh status: PASS host: vmi3080463 (VPS 38.242.240.89) db_engine: PostgreSQL 16 (container postgres) db: directus schema: sandbox_tac secret_hygiene: scanned — Kuma push token masked as <KUMA_PUSH_TOKEN>

Backup Fix Execution Log — Option A (GRANT) + Option D (Hardening)

Section 0 — Execution Authority

  • Mode: governed AI execution gate (not manual human psql).
  • Authority chain: GPT R20 fix-path decision → GPT gate review R20+R21+R22 PASS with patches applied → User authorized via gate doc → Claude Code (Opus 4.7, effort=high) executed.
  • Excluded paths (per gate H): G6 retry, PF-07 patch, Directus mutation, G8/G11/P9. Not touched.
  • Compliance: Đ32 (DDL gate), Đ33 (script-edit gate), Đ35 (DOT 100% — coverage gap declared, see §7), Đ39 (schema permissions canonical), Hiến pháp (100% DOT/AI, no manual psql).

Section 1 — Pre-Checks (10 items)

# Check Evidence Result
1 SSH contabo connectivity, hostname is VPS hostname=vmi3080463, Linux ... 6.8.0-90-generic, UTC 2026-04-27 14:54:40 PASS
2 Schema sandbox_tac exists in directus DB pg_namespace.nspname='sandbox_tac' returned 1 row; 8 tables, 0 sequences in schema PASS
3 has_schema_privilege('directus','sandbox_tac','USAGE') = false usage:false, create:false PASS
4 sandbox_tac has 0 Directus collection bindings SELECT COUNT(*) FROM directus_collections WHERE collection IN (... pg_tables WHERE schemaname='sandbox_tac') → 0 PASS
5 Backup script identified, owner & mode safe /opt/incomex/scripts/pg-backup.sh mode=755 owner=root:root size=2157 (not world-writable) PASS
6 Pre-edit SHA-256 + snapshot recorded SHA-256 db94443485291db2019fc1619df89f661cafe694b4dbdba82ee31f89433c1214; snapshot copy at /opt/incomex/scripts/pg-backup.sh.pre-fix-2026-04-27 (same SHA confirmed) PASS
7 Last-good baseline recorded directus_2026-04-26_0000.sql.gz size=43,763,755 bytes (~42 MB) at 2026-04-26 02:00; size growth trend 39→42 MB over 11 days PASS
8 DOT/OR pre-check: DOT for PG GRANT + script hardening DOT coverage gap. No DOT registered for PG GRANT operations or backup-script hardening in DOT registry. Proceeding under governed AI execution gate per Đ35 fallback clause; gap logged here. PASS (gap declared)
9 Baseline privilege capture (revoke target) Pre-GRANT: tables_priv=0, seq_priv=0, default_priv=0 (pg_default_acl for sandbox_tac namespace = 0). Rollback target = revoke exactly what gate granted. PASS
10 Script patch pre-checks (no concurrency, disk, syntax, owner/mode) pgrep pg-backup|pg_dump → none; /dev/sda1 96G used=42G avail=55G use=44%; bash -n original PASS; owner/mode captured (see #5) PASS

1.x Root cause confirmed (from backup.log tail)

[2026-04-27T00:00:01Z] START pg-backup -> /opt/incomex/backups/pg/directus_2026-04-27_0000.sql.gz
pg_dump: error: query failed: ERROR:  permission denied for schema sandbox_tac
pg_dump: detail: Query was: LOCK TABLE ... sandbox_tac.section_type_vocab, sandbox_tac.publication_type_vocab, ... IN ACCESS SHARE MODE
[2026-04-27T13:53:48Z] START pg-backup -> /opt/incomex/backups/pg/directus_2026-04-27_1353.sql.gz
pg_dump: error: query failed: ERROR:  permission denied for schema sandbox_tac

Two 20-byte gz files (Apr 27 02:00 and 13:53) confirm silent failure path: set -e killed the script after pg_dump|gzip pipefail BEFORE the size-check rm could fire, leaving the partial gz at the visible BACKUP_FILE path. This validates Mutation D's temp+atomic+trap design.

Section 2 — Mutation A: Narrow GRANT

2.1 Statements executed

GRANT USAGE ON SCHEMA sandbox_tac TO directus;
GRANT SELECT ON ALL TABLES IN SCHEMA sandbox_tac TO directus;
GRANT USAGE, SELECT ON ALL SEQUENCES IN SCHEMA sandbox_tac TO directus;

Issued via:

docker exec postgres psql -U workflow_admin -d directus -v ON_ERROR_STOP=1 \
  -c '<grant1>' -c '<grant2>' -c '<grant3>'

(Initial heredoc attempt yielded silent no-op due to SSH stdin handling; switched to -c multi-statement form. Heredoc no-op verified by re-querying privileges — no partial state.)

2.2 PG response

GRANT
GRANT
GRANT

2.3 Verify (post-GRANT)

Check Expected Actual
has_schema_privilege('directus','sandbox_tac','USAGE') true true
has_schema_privilege('directus','sandbox_tac','CREATE') false (no over-grant) false
SELECT priv on tables in schema (count) 8 8

Result: PASS. Narrow grant only — no CREATE, no OWNER change, no search_path change.

Section 3 — Mutation D: Script Hardening

3.1 Hardening summary

Concern Old behavior New behavior
Partial-file persistence pg_dump|gzip > $BACKUP_FILE direct; on set -e exit, partial 20-byte file remained at visible path Write to $TMP_FILE (.tmp.directus_*.$$.sql.gz); atomic mv -f only after all validations PASS
Cleanup on failure None (size-check rm unreachable when set -e fired earlier) trap cleanup EXIT INT TERM HUP quarantines partial temp to quarantine/ with rc suffix, or rms
Size floor 1024 bytes (gz of empty pg_dump can pass) 1,048,576 bytes (1 MB hard floor; legitimate dumps are tens of MB)
Content validation gunzip -t only gunzip -t + header regex ^-- PostgreSQL database dump + body regex ^(CREATE TABLE|COPY ...)
Failure alert stderr only (silent except cron MAIL) fail_alert() pushes Kuma status=down + stderr + non-zero exit
Concurrency None flock -n 9 on /var/lock/pg-backup.lock
Heartbeat write order After backup write (no validation gate before) Only after atomic move + all validations PASS
Curl timeouts None --max-time 10 on all Kuma pushes
find retention scope Recursive (could delete quarantine) -maxdepth 1 (quarantine dir excluded)

set -euo pipefail retained.

3.2 Diff (high-level)

Original: 2,157 bytes / 49 lines Hardened: 4,172 bytes / 117 lines (effective)

Key additions: cleanup/fail_alert functions, flock block, temp-file write path, header/body sanity check, atomic move. Kuma push URL contains push token in path component; masked as <KUMA_PUSH_TOKEN> in any external excerpts of the script.

3.3 Install + verify

install -o root -g root -m 0755 /tmp/pg-backup-hardened.sh /opt/incomex/scripts/pg-backup.sh
Check Pre Post
Owner root:root root:root
Mode 755 755
bash -n PASS PASS
SHA-256 (snapshot) db94443485291db2019fc1619df89f661cafe694b4dbdba82ee31f89433c1214 (preserved at pg-backup.sh.pre-fix-2026-04-27)
SHA-256 (live) 9d02e674752153d55084c0a05a555e190f3e9cd7f939a712e726f33891045b21 (matches Mac→VPS transfer SHA)

Section 4 — Post-Verify (8 items)

# Check Evidence Result
1 has_schema_privilege('directus','sandbox_tac','USAGE') = true usage:true PASS
2 has_schema_privilege('directus','sandbox_tac','CREATE') = false create:false PASS
3 Governed backup script exit 0 --- exit: 0 after run PASS
4 Backup size ≥ 1 MB and comparable to baseline directus_2026-04-27_1459.sql.gz = 44,712,547 bytes (~43 MB) vs baseline 43,763,755 (~42 MB); same order of magnitude, slightly larger consistent with 1-day data growth PASS
5 gzip valid gunzip -t exit 0 PASS
6 PG dump header + CREATE TABLE/COPY present -- PostgreSQL database dump header found; ^CREATE TABLE count = 219; ^COPY count = 219; sandbox_tac.* references count = 77 (proves previously-failing schema is now dumped) PASS
7 No new 20-byte files New file 44,712,547 bytes; pre-existing 20-byte files (Apr 27 02:00 and 13:53) retained per gate H#10 (no deletion of old backups) — they will age out via 7-day retention PASS
8 Failure-path verify (temp/quarantine separation) /opt/incomex/backups/pg/quarantine/ exists, empty (no failure since hardening); no .tmp.* remnants in BACKUP_DIR PASS

4.x Successful run output

[2026-04-27T14:59:56Z] START pg-backup -> /opt/incomex/backups/pg/directus_2026-04-27_1459.sql.gz
[2026-04-27T15:00:27Z] OK size=43M (44712547 bytes) header_hits=1 body_hits>=1
[2026-04-27T15:00:27Z] RETENTION kept=10 max_days=7
[2026-04-27T15:00:28Z] HEARTBEAT sent to Kuma

(kept=10 includes the two old 20-byte failure files; not deleted per gate H#10.)

Section 5 — Rollback Posture

  • Mutation A rollback (baseline-aware, per Pre-check #9 capture):
    REVOKE USAGE, SELECT ON ALL SEQUENCES IN SCHEMA sandbox_tac FROM directus;REVOKE SELECT ON ALL TABLES IN SCHEMA sandbox_tac FROM directus;REVOKE USAGE ON SCHEMA sandbox_tac FROM directus;
    
    Safe: baseline tables_priv/seq_priv/default_priv all = 0, so REVOKE returns role to exact pre-state.
  • Mutation D rollback: install -o root -g root -m 0755 /opt/incomex/scripts/pg-backup.sh.pre-fix-2026-04-27 /opt/incomex/scripts/pg-backup.sh. Snapshot SHA db944434… preserved on host.
  • Backup files: old files NOT touched (per gate H#10 + #7 verify).

Section 6 — Hard Exclusions Compliance (gate Section H)

# Forbidden Status
1 DROP schema sandbox_tac NOT performed
2 GRANT CREATE NOT performed (verified create=false post-GRANT)
3 ALTER OWNER / ALTER search_path NOT performed
4 Directus roles/collections/permissions mutation NOT performed
5 Writes to taxonomy_labels / entity_labels / system_issues / dot_action_log NOT performed
6 Cron / systemd / rclone destination change NOT performed
7 Logging unmasked secrets/tokens Kuma push token masked in this log as <KUMA_PUSH_TOKEN>
8 git commit / git push NOT performed
9 G6 retry / PF-07 patch / G8 / G11 / P9 NOT performed
10 Deleting old backups NOT performed (old 20-byte files retained)
11 Manual human psql NOT used (governed AI gate execution only)

Section 7 — DOT Coverage Gap (Đ35 follow-up)

  • Gap: No DOT exists in DOT registry for:
    • dot-pg-grant-narrow (PostgreSQL narrow GRANT operation under governance)
    • dot-backup-script-harden (backup-script hardening with temp+atomic+trap pattern)
  • Action this gate: proceeded under governed AI execution gate (User+GPT authorized) per Đ35 fallback clause, gap logged here.
  • Recommended follow-up (separate session, NOT this gate): propose DOT specs for both, route via APR.
  • No registry mutation performed in this gate to honor H#5 (no system_issues writes).

Section 8 — Verdict

PASS. Backup integrity restored.

  • Mutation A: USAGE granted, no over-grant (CREATE=false). 8 tables SELECTable, schema sandbox_tac now dumpable by directus role.
  • Mutation D: temp+atomic+trap+content-sanity in place. Failure path now actively alerts (Kuma down + non-zero exit) and quarantines partial files away from good backups.
  • Post-run backup directus_2026-04-27_1459.sql.gz is 43 MB, gzip valid, 219 CREATE TABLE / 219 COPY / 77 sandbox_tac references — full content restored.

Section 9 — STOP

Per gate Section I:

  1. Action log uploaded.
  2. STOP.
  3. NO G6 retry from this executor.
  4. NO PF-07 patch from this executor.
  5. Awaiting GPT/User review.
  6. If review confirms: next gate sequence is PF-07 v0.5 → wrapper v0.6 → G6 retry authorization.

End of log — Backup Fix Execution Gate A+D — 2026-04-27 — Claude Code executor — high effort.