Incomex AI Portal
KB-292A

S173 — GH Push RCA: Architecture Dependency

11 min read Revision 1
reports173rcagit-pushiamgcparchitecture2026-04-08

S173 — GitHub Push RCA (Root Cause Analysis)

Date: 2026-04-08 | Agent: Claude Code (Opus 4.6) | READ-ONLY

A. Preflight (P0 + P0.5)

P0: gcloud auth + capabilities

ACTIVE  ACCOUNT
*       cursor-ci-builder@github-chatgpt-ggcloud.iam.gserviceaccount.com

Project: github-chatgpt-ggcloud
Permission Status
projects.list ✅ HAS
iam.serviceAccounts.get (self-describe) ✅ HAS
secretmanager.versions.access ❌ DENIED
secretmanager.secrets.list ❌ DENIED
secretmanager.secrets.getIamPolicy ❌ DENIED
iam.serviceAccounts.list ❌ DENIED
projects.getIamPolicy ❌ DENIED
logging.read ❌ API NOT ENABLED

P0 result: SA can self-describe + list projects. CANNOT read IAM policies, secrets, or audit logs. V2 (IAM audit), V3 (SA IAM policy), V4 (secret IAM policy) are BLOCKED — evidence below limited to what this SA can see.

P0.5: Verify identifiers

Project: github-chatgpt-ggcloud (name: Github-chatgpt) ✅
Secret name: CANNOT LIST (permission denied) — known from script: gh_pat_sync_secrets
SA email: cursor-ci-builder@github-chatgpt-ggcloud.iam.gserviceaccount.com ✅ (describe OK)

B. Timeline (V1 + V7)

Full log (36 lines total, 3 failures):

2026-04-05 16:00:02 UTC  Starting daily backup
2026-04-05 16:00:16 UTC  Done. Both repos pushed to branch vps-daily-20260405
2026-04-06 04:00:01 UTC  Starting daily backup
2026-04-06 04:00:11 UTC  Done. Both repos pushed to branch vps-daily-20260406
2026-04-06 16:00:01 UTC  Starting daily backup
2026-04-06 16:00:28 UTC  FATAL: no token             ← FIRST FAILURE
2026-04-07 04:00:01 UTC  Starting daily backup
2026-04-07 04:00:28 UTC  FATAL: no token
2026-04-07 16:00:02 UTC  Starting daily backup
2026-04-07 16:00:25 UTC  FATAL: no token

Token cache timeline:

Token file: /opt/incomex/.git-token
  mtime: 2026-04-05 16:56:53 CEST (= 14:56:53 UTC)
  size: 41 chars (ghp_b7qpVH...)
  mode: 600 root:root
  TTL: 86400 seconds (24h) — hardcoded in script
Cron run (UTC) Token age > 24h? Result
04-05 16:00 fresh (just set by someone) NO ✅ SUCCESS
04-06 04:00 13h 4m (46,800s) NO ✅ SUCCESS (cache valid)
04-06 16:00 25h 4m (90,240s) YES 🔴 FAIL (cache expired → GSM denied)
04-07 04:00 37h 4m YES 🔴 FAIL
04-07 16:00 49h 4m YES 🔴 FAIL

Window of failure onset: Between 2026-04-06 04:00 UTC (last success) and 2026-04-06 16:00 UTC (first failure). Cause: cache TTL 24h expired, GSM fallback returns PERMISSION_DENIED.

C. Root Cause (1 sentence + evidence)

Root cause: SA cursor-ci-builder does NOT have secretmanager.versions.access permission on secret gh_pat_sync_secrets, so when the cached GitHub PAT token expires (24h TTL), the script cannot refresh it from Google Secret Manager and fails with "FATAL: no token".

Evidence:

$ gcloud secrets versions access latest --secret="gh_pat_sync_secrets"
ERROR: PERMISSION_DENIED: Permission 'secretmanager.versions.access' denied
  on resource. Authenticated as cursor-ci-builder@...iam.gserviceaccount.com

$ gcloud iam service-accounts describe cursor-ci-builder@...
displayName: GitHub Actions Build / Push Artifact Registry
email: cursor-ci-builder@github-chatgpt-ggcloud.iam.gserviceaccount.com
projectId: github-chatgpt-ggcloud

V5: Chu SA (critical for classification)

displayName: "GitHub Actions Build / Push Artifact Registry"
email: cursor-ci-builder@github-chatgpt-ggcloud.iam.gserviceaccount.com
projectId: github-chatgpt-ggcloud
  • displayName noi ro: SA tao cho GitHub Actions CI/CD (build + push artifact registry)
  • KHONG tao cho VPS cron job. VPS "muon" SA nay de doc secret
  • createTime: UNCLEAR — describe khong tra createTime, Logging API not enabled nen khong trace CreateServiceAccount event
  • KHONG XAC DINH ai/khi nao revoke permission — can GCP project admin dang nhap Console de check IAM audit log

D. Phan Loai: KIEN TRUC (khong phai van hanh)

Evidence Ket luan
SA name cursor-ci-builder Tao cho CI/CD, khong cho VPS
displayName "GitHub Actions Build / Push Artifact Registry" XAC NHAN CI/CD purpose
VPS cron script phu thuoc SA nay de doc GSM Phu thuoc cheo: VPS → CI/CD SA → GSM
SA permission bi revoke NGOAI VPS scope VPS khong co quyen/kha nang tu khoi phuc
Logging API not enabled Khong trace duoc ai/khi nao thay doi

Phan loai: KIEN TRUC. VPS backup pipeline phu thuoc credential do ngoai VPS quan ly (CI/CD SA). Bat ky thay doi IAM nao tren project GCP deu co the lam VPS push fail ma VPS khong biet, khong tu fix duoc. Day KHONG phai config drift (van hanh) — day la dependency architecture vulnerability.

E. Phuong An Khac Phuc

PA-1: Patch tam — refresh .git-token thu cong (KHONG vinh vien)

  • Huyen dang nhap GH → tao PAT moi → ghi vao /opt/incomex/.git-token
  • CQ-1: KHONG — PAT het han lai fail. Token cache TTL 24h cung khong doi
  • NT-02: KHONG — can nguoi refresh tay
  • NT-05: KHONG — khong tu phat hien PAT het han
  • Danh gia: chi de unblock ngay, KHONG dung lam solution

PA-2: Fix GSM permission — grant secretAccessor cho SA hien tai

  • GCP admin: gcloud secrets add-iam-policy-binding gh_pat_sync_secrets --member="serviceAccount:cursor-ci-builder@..." --role="roles/secretmanager.secretAccessor"
  • CQ-1: KHONG — SA CI/CD van bi quan ly ngoai VPS. Permission co the bi revoke lai
  • NT-02: CO — sau khi grant, script tu dong hoat dong
  • NT-05: KHONG — khong biet khi nao bi revoke lan nua
  • Danh gia: fix nhanh nhung khong cat phu thuoc

PA-3: Tao SA rieng cho VPS (CAT PHU THUOC — KHUYEN NGHI)

  • Tao vps-ops@github-chatgpt-ggcloud.iam.gserviceaccount.com
  • Grant CHI secretmanager.versions.access tren gh_pat_sync_secrets
  • Cai dat key tren VPS, doi gcloud auth sang SA moi
  • CQ-1: CO — SA rieng, lifecycle do VPS team quan ly
  • NT-02: CO — script khong doi logic, chi doi credential
  • NT-05: PHAN — can them monitoring (DOT-GIT-SYNC-MONITOR)
  • Danh gia: cat phu thuoc CI/CD SA. TOI UU

PA-4: Khong dung GSM — luu PAT truc tiep tren VPS

  • Ghi GitHub PAT vao /opt/incomex/.git-token (chmod 600), bo logic GSM trong script
  • CQ-1: PHAN — PAT van het han (90 ngay default GitHub)
  • NT-02: CO (trong thoi han PAT) — khong phu thuoc GSM
  • NT-05: KHONG — khong biet khi nao PAT het han
  • Danh gia: don gian nhung kem an toan + PAT rotation thu cong

PA-5: GitHub App token thay PAT (TOI UU NHAT nhung phuc tap)

  • Tao GitHub App → installation token auto-rotate (1h TTL, tu tao lai)
  • Khong can GSM, khong can PAT, khong can SA
  • CQ-1: CO — auto-rotate, khong phu thuoc GCP
  • NT-02: CO — app token tu tao moi request
  • NT-05: CO — GitHub App health check kha thi
  • Danh gia: toi uu nhat nhung setup phuc tap hon. Long-term.

Tong hop

PA Vinh vien (CQ-1) Tu dong (NT-02) Tu phat hien (NT-05) Do phuc tap Khuyen nghi
PA-1 Patch ❌ ❌ ❌ Thap Chi de unblock
PA-2 Fix GSM ❌ ✅ ❌ Thap Ngan han
PA-3 SA rieng ✅ ✅ Phan Trung binh KHUYEN NGHI
PA-4 PAT truc tiep Phan ✅ ❌ Thap Fallback
PA-5 GitHub App ✅ ✅ ✅ Cao Long-term

F. Branch "goc ngoai VPS scope"

Root cause KHONG fix duoc tu VPS:

  • VPS khong co quyen thay doi IAM tren GCP project
  • VPS khong co quyen enable Cloud Logging API
  • VPS khong co quyen tao SA moi

WHO co tham quyen: GCP project owner/admin (Huyen hoac team quan ly project github-chatgpt-ggcloud).

De xuat cat phu thuoc (PA-3): Tao SA rieng vps-ops@... voi minimum permissions, do VPS team quan ly. Decouples VPS tu CI/CD pipeline. Ngay ca khi CI/CD SA bi thay doi, VPS van push duoc.

G. UNCLEAR + buoc dieu tra tiep

Muc UNCLEAR Ly do Buoc tiep
Khi nao SA mat quyen UNCLEAR Logging API not enabled + SA khong co logging.read GCP admin: enable Logging API + check IAM audit log
Ai revoke quyen UNCLEAR Khong doc duoc IAM audit GCP admin: Console → IAM → Activity tab
SA co duoc Cursor IDE tao khong UNCLEAR Ten "cursor-ci-builder" goi y nhung khong xac nhan. displayName "GitHub Actions Build / Push Artifact Registry" goi y GH Actions, khong phai Cursor IDE GCP admin: check SA creation log hoac Cursor settings
PAT ghp_b7qpVH... con valid khong UNCLEAR Khong test duoc (READ-ONLY constraint) Huyen: test curl -H "Authorization: token ghp_..." https://api.github.com/user
SA co bao nhieu quyen con lai UNCLEAR Khong doc duoc project IAM policy GCP admin: gcloud projects get-iam-policy github-chatgpt-ggcloud

H. TU KIEM

Muc Status Note
CP-01 SKILL+OR+CONFIRM ✅ CHECKPOINT 1-5 DONE printed
CP-04 Assembly ✅ GCP → script → log → evidence
CP-05 search ✅ S171C report read
CP-06 reports ✅ s173-gh-push-rca created
CP-13 tu doc ✅ Script + log read
CP-14 gcloud approved ✅ Desktop approved, READ-ONLY
CP-16 raw evidence ≥5 dong ✅ Full log, SA describe, permissions table
CQ-2 verify identifiers P0.5 ⚠️ Project verified. Secret name from script (cant list). SA verified via describe
NT-09 UNCLEAR ✅ 5 UNCLEAR items with reasons
NT-11 ngan ✅
§IX AP-OUT-OF-SCOPE ✅ IAM blocked → reported, not patched
§0-AE AP-BAO-XUONG ✅ Raw output included
§0-AQ ✅ No containers touched
READ-ONLY ✅ No writes to IAM/secrets/scripts/cron

KHONG DAT:

  • V2 IAM audit: BLOCKED — Logging API not enabled + SA lacks permission
  • V3 SA IAM policy: BLOCKED — SA lacks iam.serviceAccounts.getIamPolicy
  • V4 Secret IAM policy: BLOCKED — SA lacks secretmanager.secrets.getIamPolicy
  • V5 SA createTime: UNCLEAR — describe doesn't return createTime, no audit logs

S173-RCA DONE. Root cause: KIEN TRUC — VPS phu thuoc CI/CD SA khong co quyen Secret Manager. Khuyen nghi PA-3 (SA rieng).