10 — MARK/CUT Pipeline → Queue Mapping
10 — MARK/CUT Pipeline → Queue Mapping
DESIGN-ONLY. Existing operator aliases unchanged. Existing staging lifecycle unchanged. Mapping is additive.
§1. Goal
Show how the current operator-driven cutting flow (copy_to_staging → mark → verify_mark → cut → verify_cut → cleanup) is expressed on the DP2 job_queue substrate without modifying any existing function signature, table, or constraint.
The mapping has two readings:
- Reading 1 (operator-driven, today's behaviour): each step still calls
fn_iu_op_<step>directly; the queue is an optional audit + retry surface. - Reading 2 (puller-driven, optional): a puller (Agent / Hermes / DOT command) advances queued steps automatically; the operator only intervenes for APPROVE.
Both readings coexist; the choice is per-collection or per-event_type and lives in consumer_registry (DP5).
§2. Current cutting flow (recap, unchanged)
| # | Step | Operator alias | Underlying primitive | Status after |
|---|---|---|---|---|
| 0 | source pull | (filesystem) | n/a | source_exists |
| 1 | copy_to_staging | (currently in MARK) | implicit | copied |
| 2 | mark | fn_iu_op_mark_file |
fn_iu_mark_create_manifest |
iu_staging_record.lifecycle_status='pending_review' |
| 3 | verify_mark | fn_iu_op_verify_mark |
fn_iu_verify_mark |
lifecycle_status='approved' (when verdict=approved + apply) |
| 4 | approve | (decision, implicit in verify_mark apply) | gate approved_at NOT NULL AND approval_doc_id NOT NULL |
(no status change; gates open) |
| 5 | cut | fn_iu_op_cut |
fn_iu_cut_from_manifest (G1–G7) |
lifecycle_status='consumed' |
| 6 | verify_cut | fn_iu_op_verify_cut |
fn_iu_verify_cut_result (axes A/B/C) |
verdict written; no staging status change |
| 7 | cleanup (15d→30d) | fn_iu_op_cleanup_dry_run then real |
fn_iu_staging_cleanup |
expired/cleaned |
No change to the above. Operator aliases keep their signature. Existing iu_core.iu_staging_record schema untouched.
§3. Queue mapping
Each step becomes one job_queue row whose executor is a registered §11.5 actor and whose payload_ref is the staging artifact.
§3.1 Job kinds
job_kinds_for_mark_cut:
- copy_to_staging # may be omitted if mark fn does the copy as today
- mark # wraps fn_iu_op_mark_file
- verify_mark # wraps fn_iu_op_verify_mark (mode=apply)
- approve # human/AI decision job — Agent or DOT
- cut # wraps fn_iu_op_cut
- verify_cut # wraps fn_iu_op_verify_cut
- cleanup # wraps fn_iu_op_cleanup_dry_run + real cleanup
§3.2 Status map (cutting → §6.7 work state machine)
| Cutting milestone | iu_staging_record.lifecycle_status |
job_queue.status |
|---|---|---|
| Mark job enqueued | — | queued |
| Mark in progress | — | leased then in_progress |
| Mark succeeded | pending_review |
succeeded |
| Verify_mark in progress | pending_review |
in_progress |
| Verify_mark succeeded | approved |
succeeded |
| Approve in progress | approved |
in_progress |
| Approve succeeded | approved (+ approval_doc_id set) |
succeeded |
| Cut in progress | approved |
in_progress |
| Cut succeeded | consumed |
succeeded |
| Verify_cut succeeded | consumed |
succeeded |
| Cleanup 15d | expired |
succeeded |
| Cleanup 30d | cleaned |
succeeded |
Two lifecycle vocabs (data-tier and work-tier) per §6.7 reminder. The mapping above honours that.
§3.3 Idempotency keys per kind
mark idempotency_key = source_hash
verify_mark idempotency_key = staging_record_id
approve idempotency_key = staging_record_id||':approve'
cut idempotency_key = staging_record_id||':cut'
verify_cut idempotency_key = run_id (from cut fn return)
cleanup idempotency_key = staging_record_id||':cleanup'
copy_to_staging idempotency_key = source_path||':'||source_hash
§3.4 Executor assignment
mark executor = Agent | Codex | DOT (depending on operator preference)
verify_mark executor = DOT
approve executor = Agent (Opus / GPT) — human-in-the-loop seam
cut executor = DOT
verify_cut executor = DOT
cleanup executor = PG_worker
copy_to_staging executor = DOT or external_worker
Each executor must hold a job_subscription row (DP6) covering the relevant job_kind + event_domain='staging'.
§4. Event tie-in
Each successful step emits the appropriate event into event_outbox (using existing fn_iu_emit_event):
mark succeeded → staging.record_created (already registered)
verify_mark succeeded → staging.record_approved (already registered)
cut succeeded → staging.record_consumed (already registered)
cleanup succeeded → staging.record_cleaned (already registered)
verify_cut succeeded → (no separate event — observable via dot_iu_command_run)
No new event_type needed at Phase 1 for MARK/CUT. The 5 staging.* event_type_registry rows already exist; today they have 0 emissions because the mutator-side hook is wired but the workflow is small.
After DP5 consumer_registry is live, each event_type can route to a next-step job_kind. Example: staging.record_approved → enqueue cut job for the same staging_record_id (puller-driven Reading 2).
§5. Operator-driven vs puller-driven
| Aspect | Reading 1 (operator) | Reading 2 (puller) |
|---|---|---|
| Who calls fn_iu_op_<step> | Operator directly | Worker claims job_queue row + calls fn |
| Where the next step lives | Operator memory / playbook | consumer_registry config |
| When approve is required | Always | Always; approve is the explicit human-in-the-loop job |
| Failure mode | Operator retries | DP3 retry/DLQ |
| Audit | dot_iu_command_run |
dot_iu_command_run + job_queue row |
Reading 2 is opt-in per-collection via a dot_config flag (queue.cut_pipeline.puller_enabled.<collection_id>), defaulting to false. The current operator flow keeps working unchanged.
§6. Tables / views (none new)
This mapping introduces no new table beyond DP2/DP5/DP6 substrate. Existing surfaces:
iu_core.iu_staging_record— untouched.iu_core.iu_staging_payload— untouched.dot_iu_command_run— keeps its ledger role.event_outbox(staging.* events) — already wired.
New (from DP2-6) consumed here:
job_queuerows taggedevent_domain='staging'.consumer_registryrows for eachstaging.*event_type → nextjob_kind.job_subscriptionrows for the executors named in §3.4.
§7. Compatibility with Điều 45 v1.0
| Clause | Compliance |
|---|---|
| §3.3 transitional fn_iu_* names | ✅ no rename |
| §6.7 work state machine | ✅ mapped per §3.2 |
| §11.5 executor whitelist | ✅ per §3.4 |
| §13.4 MOT NOT executor | ✅ MOT does not appear in §3.4 |
| §14 NVSZ | ✅ staging payload never enters vector tier (preserved) |
| §18.4 sub-design packs | ✅ this is the DP10-equivalent (cutting flow puller) — survey D10 question |
§8. Implementation prerequisites
- DP2 (
job_queue), DP5 (consumer_registry), DP6 (job_subscription) all in place. - No change to
fn_iu_op_*— they become executor entry-points unchanged. - One worker registered as executor for each kind (Agent / Codex / DOT / PG_worker).
§9. Rollback
- Reading 1 (operator-driven) remains the always-available fallback — even if all Reading 2 config is disabled, the operator can still call aliases directly.
- Per-collection puller toggle returns the pipeline to operator-driven instantly.
§10. Open questions
| # | Question | Routed to |
|---|---|---|
| MC-Q1 | Approve dual reading (operator + puller) co-existence? | Council |
| MC-Q2 | Which collections (if any) get puller enabled at Phase 5 pilot? | Council + operator |
| MC-Q3 | Should approve job exist as a real job_kind (vs human action outside queue)? |
Council |
| MC-Q4 | Should copy_to_staging become a separate job (currently inlined in mark)? |
Council |
| MC-Q5 | Should verify_cut advance any status, or remain verdict-only? |
Council |
Cross-cutting mapping. No mutation. Authored 2026-05-26.