KB-3A36

10 — MARK/CUT Pipeline → Queue Mapping

9 min read Revision 1
design-packdieu-45mark-cutqueue-mappingiu-stagingoperator-aliasdesign-only

10 — MARK/CUT Pipeline → Queue Mapping

DESIGN-ONLY. Existing operator aliases unchanged. Existing staging lifecycle unchanged. Mapping is additive.


§1. Goal

Show how the current operator-driven cutting flow (copy_to_staging → mark → verify_mark → cut → verify_cut → cleanup) is expressed on the DP2 job_queue substrate without modifying any existing function signature, table, or constraint.

The mapping has two readings:

  • Reading 1 (operator-driven, today's behaviour): each step still calls fn_iu_op_<step> directly; the queue is an optional audit + retry surface.
  • Reading 2 (puller-driven, optional): a puller (Agent / Hermes / DOT command) advances queued steps automatically; the operator only intervenes for APPROVE.

Both readings coexist; the choice is per-collection or per-event_type and lives in consumer_registry (DP5).


§2. Current cutting flow (recap, unchanged)

# Step Operator alias Underlying primitive Status after
0 source pull (filesystem) n/a source_exists
1 copy_to_staging (currently in MARK) implicit copied
2 mark fn_iu_op_mark_file fn_iu_mark_create_manifest iu_staging_record.lifecycle_status='pending_review'
3 verify_mark fn_iu_op_verify_mark fn_iu_verify_mark lifecycle_status='approved' (when verdict=approved + apply)
4 approve (decision, implicit in verify_mark apply) gate approved_at NOT NULL AND approval_doc_id NOT NULL (no status change; gates open)
5 cut fn_iu_op_cut fn_iu_cut_from_manifest (G1–G7) lifecycle_status='consumed'
6 verify_cut fn_iu_op_verify_cut fn_iu_verify_cut_result (axes A/B/C) verdict written; no staging status change
7 cleanup (15d→30d) fn_iu_op_cleanup_dry_run then real fn_iu_staging_cleanup expired/cleaned

No change to the above. Operator aliases keep their signature. Existing iu_core.iu_staging_record schema untouched.


§3. Queue mapping

Each step becomes one job_queue row whose executor is a registered §11.5 actor and whose payload_ref is the staging artifact.

§3.1 Job kinds

job_kinds_for_mark_cut:
  - copy_to_staging          # may be omitted if mark fn does the copy as today
  - mark                     # wraps fn_iu_op_mark_file
  - verify_mark              # wraps fn_iu_op_verify_mark (mode=apply)
  - approve                  # human/AI decision job — Agent or DOT
  - cut                      # wraps fn_iu_op_cut
  - verify_cut               # wraps fn_iu_op_verify_cut
  - cleanup                  # wraps fn_iu_op_cleanup_dry_run + real cleanup

§3.2 Status map (cutting → §6.7 work state machine)

Cutting milestone iu_staging_record.lifecycle_status job_queue.status
Mark job enqueued queued
Mark in progress leased then in_progress
Mark succeeded pending_review succeeded
Verify_mark in progress pending_review in_progress
Verify_mark succeeded approved succeeded
Approve in progress approved in_progress
Approve succeeded approved (+ approval_doc_id set) succeeded
Cut in progress approved in_progress
Cut succeeded consumed succeeded
Verify_cut succeeded consumed succeeded
Cleanup 15d expired succeeded
Cleanup 30d cleaned succeeded

Two lifecycle vocabs (data-tier and work-tier) per §6.7 reminder. The mapping above honours that.

§3.3 Idempotency keys per kind

mark           idempotency_key = source_hash
verify_mark    idempotency_key = staging_record_id
approve        idempotency_key = staging_record_id||':approve'
cut            idempotency_key = staging_record_id||':cut'
verify_cut     idempotency_key = run_id (from cut fn return)
cleanup        idempotency_key = staging_record_id||':cleanup'
copy_to_staging idempotency_key = source_path||':'||source_hash

§3.4 Executor assignment

mark             executor = Agent | Codex | DOT  (depending on operator preference)
verify_mark      executor = DOT
approve          executor = Agent (Opus / GPT) — human-in-the-loop seam
cut              executor = DOT
verify_cut       executor = DOT
cleanup          executor = PG_worker
copy_to_staging  executor = DOT or external_worker

Each executor must hold a job_subscription row (DP6) covering the relevant job_kind + event_domain='staging'.


§4. Event tie-in

Each successful step emits the appropriate event into event_outbox (using existing fn_iu_emit_event):

mark succeeded         → staging.record_created   (already registered)
verify_mark succeeded  → staging.record_approved  (already registered)
cut succeeded          → staging.record_consumed  (already registered)
cleanup succeeded      → staging.record_cleaned   (already registered)
verify_cut succeeded   → (no separate event — observable via dot_iu_command_run)

No new event_type needed at Phase 1 for MARK/CUT. The 5 staging.* event_type_registry rows already exist; today they have 0 emissions because the mutator-side hook is wired but the workflow is small.

After DP5 consumer_registry is live, each event_type can route to a next-step job_kind. Example: staging.record_approved → enqueue cut job for the same staging_record_id (puller-driven Reading 2).


§5. Operator-driven vs puller-driven

Aspect Reading 1 (operator) Reading 2 (puller)
Who calls fn_iu_op_<step> Operator directly Worker claims job_queue row + calls fn
Where the next step lives Operator memory / playbook consumer_registry config
When approve is required Always Always; approve is the explicit human-in-the-loop job
Failure mode Operator retries DP3 retry/DLQ
Audit dot_iu_command_run dot_iu_command_run + job_queue row

Reading 2 is opt-in per-collection via a dot_config flag (queue.cut_pipeline.puller_enabled.<collection_id>), defaulting to false. The current operator flow keeps working unchanged.


§6. Tables / views (none new)

This mapping introduces no new table beyond DP2/DP5/DP6 substrate. Existing surfaces:

  • iu_core.iu_staging_record — untouched.
  • iu_core.iu_staging_payload — untouched.
  • dot_iu_command_run — keeps its ledger role.
  • event_outbox (staging.* events) — already wired.

New (from DP2-6) consumed here:

  • job_queue rows tagged event_domain='staging'.
  • consumer_registry rows for each staging.* event_type → next job_kind.
  • job_subscription rows for the executors named in §3.4.

§7. Compatibility with Điều 45 v1.0

Clause Compliance
§3.3 transitional fn_iu_* names ✅ no rename
§6.7 work state machine ✅ mapped per §3.2
§11.5 executor whitelist ✅ per §3.4
§13.4 MOT NOT executor ✅ MOT does not appear in §3.4
§14 NVSZ ✅ staging payload never enters vector tier (preserved)
§18.4 sub-design packs ✅ this is the DP10-equivalent (cutting flow puller) — survey D10 question

§8. Implementation prerequisites

  • DP2 (job_queue), DP5 (consumer_registry), DP6 (job_subscription) all in place.
  • No change to fn_iu_op_* — they become executor entry-points unchanged.
  • One worker registered as executor for each kind (Agent / Codex / DOT / PG_worker).

§9. Rollback

  • Reading 1 (operator-driven) remains the always-available fallback — even if all Reading 2 config is disabled, the operator can still call aliases directly.
  • Per-collection puller toggle returns the pipeline to operator-driven instantly.

§10. Open questions

# Question Routed to
MC-Q1 Approve dual reading (operator + puller) co-existence? Council
MC-Q2 Which collections (if any) get puller enabled at Phase 5 pilot? Council + operator
MC-Q3 Should approve job exist as a real job_kind (vs human action outside queue)? Council
MC-Q4 Should copy_to_staging become a separate job (currently inlined in mark)? Council
MC-Q5 Should verify_cut advance any status, or remain verdict-only? Council

Cross-cutting mapping. No mutation. Authored 2026-05-26.

Back to Knowledge Hub knowledge/dev/laws/dieu44-trien-khai/v0.6-dieu45-full-queue-orchestration-design-pack/10-mark-cut-queue-mapping.md